lightning

8589 Commits

55 Branches

107 Tags

38 MiB

Tree: 7f2b332021

Commit Graph

Author	SHA1	Message	Date
Rusty Russell	daba3e7deb	common/gossmap: helper to map the gossip store. I went overboard on optimization. I am so sorry: 1. Squeezed channel min/max into 16 bits. 2. Uses mmap and leaves node_ids in the file. 3. Uses offsets instead of pointers where possible. 4. Uses custom free-list to allocate inside arrays. 5. Ignores our autogenerated marshalling code in favor of direct derefs. 6. Carefully aligns everything so we use minimal ram. The result is that the current gossip_store: - load time (-O3 -flto laptop): 40msec - load time (-g laptop i.e. DEVELOPER=0): 60msec - load time (-O0 laptop i.e. DEVELOPER=1): 110msec - Total memory: 2.6MB: - 1.5MB for the array of channels - 512k for the channel htable to map scid -> channel. - 320k for the node htable to map nodeid -> node. - 192k for the array of channels inside each node - 94k for the array of nodes Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>	5 years ago

Author

SHA1

Message

Date

Rusty Russell

daba3e7deb

common/gossmap: helper to map the gossip store.

I went overboard on optimization.  I am so sorry:
1. Squeezed channel min/max into 16 bits.
2. Uses mmap and leaves node_ids in the file.
3. Uses offsets instead of pointers where possible.
4. Uses custom free-list to allocate inside arrays.
5. Ignores our autogenerated marshalling code in favor of direct derefs.
6. Carefully aligns everything so we use minimal ram.

The result is that the current gossip_store:
 - load time (-O3 -flto laptop): 40msec
 - load time (-g laptop i.e. DEVELOPER=0): 60msec
 - load time (-O0 laptop i.e. DEVELOPER=1): 110msec
 - Total memory: 2.6MB:
   - 1.5MB for the array of channels
   - 512k for the channel htable to map scid -> channel.
   - 320k for the node htable to map nodeid -> node.
   - 192k for the array of channels inside each node
   - 94k for the array of nodes

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

5 years ago

1 Commits (7f2b332021b70e40452ffc2bc67954258f497580)