git ssb


cel / git-ssb-web

Feed view is slow without gitindex

Opencel-desktop opened this issue on 5/15/2020, 9:16:29 PM

Feed view is slow without gitindex

The home page feed and user feed pages are slow by default. If you have the ssb-gitindex ssb-server flume plugin, it is fast. But I assume most users won't have that.

The reason it is slow is because it does a filter of the feed/log without any index. A possible solution would be to - when ssb-gitindex is not present - instead of doing a feed/log query and filtering it, make several queries, one per message type, and merge the results using pull-many. Message types to get would be e.g. git-update, git-repo, pull-request, issue, and issue-edit.

One downside would be that issue/PR reply comments would not be visible in the feed, since I don't think there is any way to query for those with the default/common ssb-server plugins - even though git-ssb-web sets an "issue" and "repo" or "project" property on issue/PR comments to distinguish them from normal posts. But other clients don't necessarily add those extra properties, and such posts are not shown even now.

%u9+sxYAMR9sjRKfPAFC43g+G7cvTQrm2ml6MSBj6hYo=.sha256 Daan Patchwork · 1/19/2021, 9:53:10 AM

Just to clarify: why can't we include this index into git-ssb-web by default? I'd argue this actually is a block to make git-ssb-web feature complete, not a convenience addon. Scanning a 1GB offset log for git messages takes quite some time...

%G8E5So+UvdwQ0Ne2NgnKa+ZLK3uYVtmV/wrk2x5GbE0=.sha256 cel-desktop · 1/20/2021, 1:00:52 AM

@Daan git-ssb-web is a ssb client application. It has to use what plugins/functions are available in the ssb-server that it is connected to.

It may be possible to make flumelog plugins like ssb-gitindex work in a client by running a flumelog in the client backed by createLogStream. That concept was proved, I forgot by who - maybe @Christian Bundy or @arj - and I don't have a link for it right now. But that still leaves the coordination problem of which client runs the flumelog with that index, as there may be multiple simultaneously, and/or with different index versions. I think ssb-afterparty by @luxxor is supposed to solve this - I have not tried it.

It might be simpler instead to refactor the home page and feed pages to use a query per type and combine the results, as suggested in the issue. Or just show only one message type (git-update).

%luZN8iwscuuoUdjGGgHoM7a2Fp+7Ab0xwwQ07FtOdo0=.sha256 Daan Patchwork · 1/20/2021, 8:07:47 AM

Thank you @cel for clarifying.

I have long been wondering: wouldn't it be possible (if a little crude) to locate all git-ssb indexes in ~/.cache/git-ssb-index? Or, if running the entire git-ssb suite, in ~/.cache/git-ssb?

%4k9DNpj4QOzLVOfybtnzcEjSTuuJPGVmRHFpOpI9ylM=.sha256 cel-desktop · 1/21/2021, 1:27:50 AM

@Daan I assume by the entire git-ssb suite, you mean git-ssb-web. I don't see how it would help to have separate index directories here. Anyway ssb-gitindex is only used by git-ssb-web, not the git-ssb CLI or git-remote-ssb (the git remote helper).

For git-ssb to run its own indexes, we get the coordination problems I described. That is why git-ssb requires only the default ssb-server indexes. ssb-gitindex is an exception but is optional. I think the easiest solution would be to use multiple queries, but I don't know when I can implement this. Other ways could work too - but I just don't want to add another singleton e.g. making it so that only one git-ssb-web instance can run a time.

Built with git-ssb-web