git ssb

title: 'Distributed Git on Secure Scuttlebutt: The Cel Interview' date: 2016-10-04 layout: post.jade

collection: blog

<img src="/cel.jpg" class="profile full">

gb Hey @cel (pictured above, middle -- with @myf and @ev) -- I'm starting out by reviewing the code for git-ssb. You made your first commit to the repo in February 2016. What brought you to secure scuttlebutt to begin with? What was the impetus for creating git-ssb? Oh, and before we go to deep into the code, can you tell readers how we met in the first place?

cel I had a dream a day ago that you were making spaces, places where people could go. Maybe these interviews are like places...

how we met

We met in person in NYC, April 2016, at a coffee shop in NYC with @ev and @myf, where we had decided to have a meetup since we were all in the city. I had been staying at my parents' house a few hours east on Long Island, but was planning to come through the city the next day to leave for my visit to Acorn Community. I think @ev proposed we meet up, and I proposed the time and place, after asking my dad (who knows the city well) what a good place would be. So we had all been interacting on SSB before then. However, I knew of your writings and work online since before SSB. It might have been on socialno.de, the social network site on cjdns/Hyperboria, where I first encountered your identity. I read your website and joined your email list in 2013, and was one of the members of your audience. I was inspired by your writings to get the backpack you recommended and read Voluntary Simplicity. I remember when you invited people on the list to message you to say hello, but I didn't do it.

gb What brought you to secure scuttlebutt to begin with?

cel A good question, which reminds me of how @mix a while ago invited me and others to introduce ourselves. If I had done that, I would refer to it here. It is sometimes hard for me to motivate myself to do writing.

I loosely followed the evolution of secure-scuttlebutt from original/insecure scuttlebutt, and this relates to my interests going back a bit further. I wanted a platform for making web apps for peer-to-peer collaboration and communication. I wanted to make fun and useful things.

When Google Wave was in its heyday, I learned how to make real-time collaborative web apps ("gadgets") that could be embedded in threaded, editable conversations ("waves"). The most popular one I made was for free-form card games. You could move cards around in a space, flip them, shuffle them, deal them, etc., and see other players doing the same, in real-time. I later re-implemented some of it on (original) scuttlebutt, but without the surrounding social media application it wasn't as interesting. I also learned about CouchDB at some point, which allows for making standalone collaborative web apps. For my senior project in high school, I made a couchapp for collaborative maze drawing. You could go to this website and move through a big maze, and when you get out of it, you get to add on with your own drawing. In both this and the Cards gadget, I had to handle things which I later learned the theory behind, as Commutative/Conflict-free Replicatable Data Stuctures (CRDTs). This is important now for making apps on a peer-to-peer database like secure-scuttlebutt.

So I was aware of scuttlebutt, a thing for making real-time peer-to-peer things. I saw some of the other cool stuff @dominic was doing, and followed him on github. At some point I followed @paul as well. In October 2014, I saw that they were collaborating to build a decentralized social network application, phoenix, and infrastructure/database, secure-scuttlebutt. I thought this looked liked the most promising decentralized social network that I had seen, and so I gave it a try.

It was very nice. I made some minor pull requests. @paul was adding a plugin system, and I made a plugin for playing tic-tac-toe (for boards of arbitrary size, of course). Then the plugin system was removed, and I lost interest. Some few months later, I saw the app had been relaunched as Patchwork, and the database re-keyed. I joined again in October 2015, and haven't left yet. I made some more pull requests and was invited to join the Secure Scuttlebutt Consortium as a Contributor. Some months later I joined as a full member.

The core of the work was to make a "git remote helper" that would implement fetching and pushing to git repos in terms of querying and publishing messages on SSB. git makes it easy to add new remote helpers for new protocols. I "surveyed the literature" and found implementations for git remote helpers for CouchDB, IPFS, Mercurial, and Dropbox backends. These worked in different ways, and none were written in JavaScript, SSB's native language. I wanted to have access to the raw git objects in order to map cleanly between git data structures and SSB data structures, have flexibility, and make an efficient implementation. The usual ways of making a git remote helper didn't allow for that, so I decided to implement the remote helper in the most difficult way.

I ended up implementing a bunch of stuff that usually only git itself has to do, but I tried to make it abstract and reusable: this paid off when another project, Mango, was able to make use of this work for another git remote helper, based on Ethereum.

gb How'd you get started coding?

I had access to a computer before most of my peers. My parents had a Macintosh Plus which they used for their business as artists. There was no way to do programming on it, but I liked drawing things in MacPaint. When I was about 5, this computer was 8 years old, and my parents got an iMac G3. (Interestingly, these computers were so far apart in technology development that we had no way to transfer data between them, until years later when I became interested in retrieving and archiving my old files.) On the iMac I discovered AppleScript.

I learned how to create "dialog boxes" that would say whatever I wanted them to, with buttons that I could set. I made scripts to move file icons around in windows. Around this time my family got internet through AOL, and my parents decided to make a website for their business.

After rejecting an overpriced quote from a web design agency at the height of the dot-com bubble, they started making the site themselves using a WYSIWYG website maker, Adobe PageMill. I got my own section of the website, in a subdirectory. I still remember the feeling of excitement and possibility when I saw how I could add my own form elements and stuff to the pages. I put a bunch of menus and radio buttons and checkboxes (and cat pictures) on my home page just because I liked that I could do that. I was about 7 or 8 at this time.

Early website.

gb What sparked your interest in programming?

cel Along with the AppleScript, I learned a little JavaScript, by copy-pasting code snippets from javascript.com. Usually I was not able to modify them in much of a useful way because they would break with cryptic error messages, and there were these weird cross-browser incompatibilities that made things even more confusing. The best thing I had was a cat picture (which I drew in AppleWorks) that would follow the cursor as you moved it. I did not find out about server-side scripting for a while, and was confused about how websites had dynamic content. I wanted to add a search engine page, and be able to let people create accounts and log in with them, like the big websites could. I did what I now call cargo-cult programming where I tried to put a form on the page just like a real search engine had, hoping that somehow it would be able to spit out search results. Eventually I learned about CGI scripts, found some books about Perl, and made a few server-side scripts. Later I found PHP, and started doing things with that.

So, from a young age I was able to occupy myself with puzzles and pretend systems and figuring out how things worked, and that tendency found expression through computing. My parents gave me free reign on the computer as they realized I was teaching myself things.

gb Do you have a git repo you can link to of your initial code attempts?

cel My most salient initial code attempts were probably in AppleScript, figuring out how "display dialog" worked. I'll see if I can fire up an emulator and pick out some of those. Some of my PHP sites from around 2006-2008 I put into git recently - it is almost comical to see how unreadable the code style is.

gb What are your current programming challenges?

cel There are few long-standing issues with git-ssb that I have yet to tackle. I don't think I have mastered test-driven development. Doing technical writing - creating design documents, or documentation that is more than just a list of schemas and APIs, is something I could work on. The list of things I want to make grows faster than I can make them. Sometimes I spend too much time programming, thinking about programming, or doing stuff on a computer that I have a hard time falling asleep. I also haven't completely figured out how to reconcile working on the software projects that I think are valuable, with "making a living".

gb Can you talk about pull-streams?

cel I think pull-streams are really great. I like that they provide instant two-way backpressure and give you a lot of control over the flow of execution, and are basically simple. At their core there is no complex hidden logic, just an interface, and everything around them builds on that interface. You can implement a pull stream without depending on the pull-stream module, just by writing a function that follows the interface.

I learned how pull streams worked by reading through the pull-stream-examples documents. It was a little difficult to figure out at first, and even sometimes after not looking at the API for a while I have had to refresh myself on how they work. However, you usually don't have to implement a pull stream from scratch, as there are many higher-level pull-stream modules already made. If you do want to implement a pull-stream - a source, a sink, a through, or a duplex - I would suggest reading pull-stream-examples and studying any pull-stream implementation that does something similar to what you are trying to do. Or ask on SSB for help.

You can comment on this interview on sbot. Better still, give cel's code work a spin.