ssb-npm-registry

Serve an npm registry server backed by SSB.

Install

Requirement: scuttlebot running from the command line (sbot server / ssb-server start).

BLOB_ID="$(sbot links2.read --query '[{"$filter":{"rel":["mentions",{"$prefix":"npm:ssb-npm-registry:"}]}}]'|grep -o '&.*sha256'|tail -n 1)"
cd ~/.ssb && mkdir -p node_modules && cd node_modules
[ "$(sbot blobs.size "$BLOB_ID")" = null ] && sbot blobs.want "$BLOB_ID"
sbot blobs.get "$BLOB_ID" | tar xvz && mv package ssb-npm-registry
ENABLE_BLOB_ID='&9C/OST7udTFq6skK4Ix1G8zOeFaVXjpbdet1gadBnik=.sha256'
[ "$(sbot blobs.size "$ENABLE_BLOB_ID")" = null ] && sbot blobs.want "$ENABLE_BLOB_ID"
sbot blobs.get "$ENABLE_BLOB_ID" | node
# restart sbot

Develop

Install with git-ssb:

As a scuttlebot plugin:

git clone ssb://%pFqjcdVKHqsrtOjVEAVZeCF0iY4s+3Hr0vA3EFCq5UM=.sha256 ~/.ssb/node_modules/ssb-npm-registry
cd ~/.ssb/node_modules/ssb-npm-registry
wget -qO- 'http://localhost:8989/blobs/get/&E+tZfD6eodncvEddM3QAfsmzTJ003jlPGsqFN5TO7sQ=.sha256' | tar xz package/node_modules
mv package/node_modules node_modules
rmdir package
./enable.js
# restart sbot

To set ssb-npm-registry as your default registry server:

npm config set registry http://localhost:8043

Config

~/.ssb/config example:

{
  "npm": {
    "host": "localhost",
    "port": 8043,
    "autoAuth": true,
    "fetchAll": true,
    "needShasum": false,
    "baseUrl": "https://example.org",
    "defaultPkgs": ["ssb-npm"]
  }
}

• config.npm.host: hostname to listen on. default is "localhost"
• config.npm.port: hostname to listen on. default is 8043
• config.npm.autoAuth: whether to automatically update ~/.npmrc to add
• config.npm.fetchAll: whether to fetch blobs for all versions of a package, to populate package metadata, for a request to a /<package> path. Most npm clients require this. default is true
• config.npm.needShasum: whether to fetch always calculate and send package shasums. Default is false but will always calculate the shasum in response to a request from npm < v5
• config.npm.baseUrl: URL of this ssb-npm-registry instance, such as in case it is being served behind a proxy. Default is to try to detect it for each HTTP request from the Host header or local socket address.
• config.npm.defaultPkgs: Array of names (or name@version strings) of ssb-npm packages for the bootstrap snippet to install globally with npm. These are resolved to message ids when the snippet is served. Default: ["scuttlebot", "ssb-npm", "git-ssb"]
• config.npm.postInstallCmd: Command string for the bootstrap snippet to run after installing packages, if any. Default: "sbot server"

Environmental variables

• WANT_WARN_TIME: duration (ms) to wait for a blob or ooo-msg want before outputting a warning about it to stderr. Default: 60000 (1 minute). If set to a negative value, no such warnings are output.

API

var Registry = require('ssb-npm-registry')

var serve = Registry.respond(sbot, config)

Embed ssb-npm-registry into other applications using the respond method which returns a (req, res) HTTP handler function, which can be passed to a HTTP server:

var server = require('http').createServer(serve)

When using this method, the config properties npm.host, npm.port and npm.autoAuth do not have an effect, and ~/.npmrc is not modified.

Registry.publishPkgMentions(sbot, mentions, cb(err, msgs))

Publish the given npm package mentions as one or more messages.

If given exactly one package mention, this will also calculate and add dependency and version graph information to the message.

Registry.expandPkgMentions(sbot, mentions, props, cb(err, mentions))

For each package mention, fetch the tarball, extract the given properties from its package.json and add them to the mention object.

Routes

In addition to the npm registry routes, ssb-npm-registry's web server (and respond method) serve the following additional routes:

/-/bootstrap - Bootstrap shell snippet

Serves a shell script to for installing ssb-npm-registry and using it to install other ssb-npm packages by running in a bootstrap mode where it does not require running scuttlebot.

/-/blobs/get/:id - Blobs

Fetch a SSB blob by its id.

/-/msg/:id - Blobs

Fetch a SSB message by its id, as JSON.

/-/prebuild/:name - Prebuild blobs

name should be the name of a blob in the format {name}-v{version}-{runtime}-v{abi}-{platform}{libc}-{arch}.tar.gz.

This route looks up mentions for blobs with the given name prefixed by "prebuild:". If the name maps to exactly one blob, the route redirects to the local ssb-ws URL for that blob.

Example:

• npm installs leveldown v1.9.0 on node v57 on linux, arm
• prebuild-install fetches http://localhost:8043/-/prebuild/leveldown-v1.9.0-node-v57-linux-arm.tar.gz
• ssb-npm-registry serves the blob &wSehVYRREEqZhZgpMk82CHcXVNcPqXDacgj3Tit7TMc=.sha256

Message scope

Requests to the registry server can be scoped to one or more messages by prefixing the request pathname with a directory part with url-encoded message ids. You can make such requests by including that prefix in the registry URL. i.e. npm --registry=http://localhost:8043/<MsgIdEnc>[,<MsgIdEnc>...]/ ...

If a request is scoped to a message, the registry server will answer the request using information contained in that message or messages that it links to (recursively), instead of by using sbot's ssb-links indexes. Fetching messages for a message-scoped request will be done using ssb-ooo if that sbot plugin is available.

A message-scoped request allows you to install a package without having to replicate the feeds of the authors of the package or its dependencies, since the information needed to install the package is referenced by message id.

If you prefix the message id with a caret (^ or %5E), the results will include packages from both the message scope and sbot's ssb-links indexes.

If sbot does not have the ssb-links scuttlebot plugin, requests to the registry must use message scope and not the caret, since ssb-links is required otherwise.

When running the ssb-npm command or a ssb-npm bootstrap server (ssb-npm-bootstrap or ssb-npm-registry/bootstrap/bin.js), the --branch
option is equivalent to adding a message id to the scope.

Bootstrapping

Since the ssb-npm and ssb-npm-registry packages are distributed on ssb-npm, there is a problem of how to get them securely if you do not already have scuttlebot installed - especially if you also want to install scuttlebot via ssb-npm. ssb-npm's bootstrap feature aims to solve this. ssb-npm-registry includes a route that serves a snippet of shell script code. Executing this bootstrap snippet fetches the ssb-npm-registry package identified by a blob id, uses that to run a bootstrap ssb-npm-registry server, and uses that to install further packages from ssb-npm (e.g. the ssb-npm command, scuttlebot, git-ssb) identified by message ids. The bootstrap snippet fetches from the network only hash-identified data (SSB blobs and messages), so the package contents cannot be modified by the server or in transit. By default, the snippet fetches content from the server that served the snippet, but it can be changed to fetch content instead from elsewhere, such as different ssb-npm-registry server, or a ssb-ws or ssb-viewer server.

License

Copyright (C) 2017-2019 Secure Scuttlebutt Consortium

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. If not, see http://www.gnu.org/licenses/.