git ssb

$ git clone ssb://%VYMDiAmQreopygieZnVpC4qUSGYzzoxwrQ6HyNCycS0=.sha256 go-muxrpc -vvvvv                                                                                                
Cloning into 'go-muxrpc'...
command: connect git-upload-pack
want 5f510d57a8743277b425b2149b072f25ed05671e (HEAD)
want 5f510d57a8743277b425b2149b072f25ed05671e (refs/heads/master)
done
line want 5f510d57a8743277b425b2149b072f25ed05671e
line want 5f510d57a8743277b425b2149b072f25ed05671e
line 
line done

Others, like e.g. @cryptix/go-shs work just fine. Anything I can do to provide more info?

Whoops, meant %/BSksWw....

I can reproduce this. That repo had some updates pushed to it before the git ssb schema was stable, so this could be an issue of waiting for unreplicated blobs, or another bug. I'll investigate later and try to provide a fix.

@keks, yes, this was because of messages in the old schema which linked to blobs in a way that didn't trigger pubs to optimistically replicate them. I've added a blob timeout now in ssb-git-repo v1.6.3 so that there will be more feedback than just hanging in the case where a blob can't be fetched.

The blobs not linked properly are the following:

&zTPQqVu... &oZ4OSCq... &thLSCN7... &EZsNpDy... &MDoaPSq... &9kWdJNJ... &/3F2XBZ... &XD0fg16...

and also these, although I have them locally since @cryptix triggered them to sync manually, but I'm not sure if the whole network will have them:

&ZXaJA0C... &/Zys/e2... &wRRdFVg... &+7o1l40... &BgNlOBy... &390u/eI... &7suLKBP... &N1xFeX5... &p/Kkojv... &5sFkngG... &QfnW4Ob... &CI4GHnm... &uhSMBVO... &9mc4w9P...

@cryptix can you publish a message mentioning these blobs? That way scuttlebot will detect links to them in your feed and sync them.

re-publish..!

&zTPQqVu... &oZ4OSCq... &thLSCN7... &EZsNpDy... &MDoaPSq... &9kWdJNJ... &/3F2XBZ... &XD0fg16... &ZXaJA0C... &/Zys/e2... &wRRdFVg... &+7o1l40... &BgNlOBy... &390u/eI... &7suLKBP... &N1xFeX5... &p/Kkojv... &5sFkngG... &QfnW4Ob... &CI4GHnm... &uhSMBVO... &9mc4w9P...

thanks @cryptix. i got those blobs now. looks like i missed one more though: &IBUHXVH.... can you mention that one too?

Huh... Here you go: &IBUHXVH...

@cryptix and this one: &3XpQgtj...

and these, just in case - interestingly, it seems the blob syncing code detected the link and synced them, but sbot.links doesn't detect the link to them (this was also in an older schema so it shouldn't affect us going forward):

&HdsYfO/... &tHMHbSc... &TBxG+NF...

According to my script that should be all to get this repo fixed:

sbot links --rel repo --dest "%VYMDiAmQreopygieZnVpC4qUSGYzzoxwrQ6HyNCycS0=.sha256" --values | jq -r '.value.content.objects | .. | select(type=="string" and .[0:1]=="&")' | while read key; do echo -n "$key "; links="$(sbot links --source "@p13zSAiOpguI9nsawkGijsnMfWmFd5rlUNpzekEE+vI=.ed25519" --dest "$key")"; echo -n "${links:+linked} "; sbot blobs.has "$key"; done

and globally (note: slow command):

sbot messagesByType --type git-update | jq -r '.. | select(type=="string" and .[0:1]=="&")' | while read key; do echo -n "$key "; sbot blobs.has "$key"; done

thanks for bearing with me on this.

Now that's some solid jq magic right there.

Here are the missing ones: &3XpQgtj... &HdsYfO/... &TBxG+NF... &tHMHbSc...

thanks for bearing with me on this.

You're more than welcome. I thought I messed something up more badly..

Would it make sense to add the query that checks if all blobs are referenced/linked to git-ssb-web as a maintanance/consistency check?

Cool, I can fetch and clone from go-muxrpc now. @keks can you confirm if it works for you now?

Would it make sense to add the query that checks if all blobs are referenced/linked to git-ssb-web as a maintanance/consistency check?

The problem of blobs being linked incorrectly shouldn't happen any more, unless someone runs old git-ssb code, or makes a new implemention that doesn't use the correct schema, or uses a stricter schema (i.e. does not implement backwards compatibility). If that happens then it might makes sense to add some checks in the UI. For now a more likely problem I think would be someone pushes an update and then disconnects before the rest of the network can get the blobs from them. For this we could have a check that shows if there are linked blobs that you don't have. It could be a page listing all the blobs referenced by a repo and whether you have each one. So far though syncing seems pretty reliable to me (other than this issue), so I don't see a big need for this, but if we start seeing more sync/fetch failures in the future then I would look into adding a diagnostic page like that.

$ git clone ssb://%VYMDiAmQreopygieZnVpC4qUSGYzzoxwrQ6HyNCycS0=.sha2> Cloning into '%VYMDiAmQreopygieZnVpC4qUSGYzzoxwrQ6HyNCycS0=.sha256'... 1% =------------------------------------------------------------------------------------------------------------2% ==-----------------------------------------------------------------------------------------------------------3% ===----------------------------------------------------------------------------------------------------------4% ====---------------------------------------------------------------------------------------------------------6% ======-------------------------------------------------------------------------------------------------------Receiving objects: 100% (105/105), 42.70 KiB | 0 bytes/s, done. Checking connectivity... done.

It works, thank you!

Great. Now to fix that progress bar...