IPFS Weekly 9

IPFS is a new hypermedia distribution protocol, addressed by content and identities, aiming to make the web faster, safer, and more open. In these posts, we highlight some of the development that has happened in the past week. For anyone looking to get involved, follow the embedded hyperlinks, search the wealth of information on GitHub or join us on IRC (#ipfs on the Freenode network).

If you would like to get this update as an email, sign up for our weekly newsletter!

Here are some of the highlights for the March 21st through March 28th sprint:

Package Managers

Package managers have been a large topic of discussion recently. Mostly, this is because of an issue with an author of some heavily used npm packages unpublishing all of his modules simultaneously. One of these was left-pad , which was used by thousands of builds globally, all of which broke when the package was removed. A great writeup of what happened is on the npm.js blog here; they took this very seriously, and shortly after changed their unpublish policy as a direct result.

Many people jumped to IPFS as a possible solution to this problem. With a permanent filesystem, unpublishing wouldn’t be possible. Here’s one post titled How to use IPFS to fix npm; here’s an issue on a new GitHub organization, ipmjs, trying to find consensus on how to fix npm using a permanent storage system; here’s an npm module, cowpen that publishes modules directly to IPFS; here’s another decentralized package manager that sprung up using IPFS and Ethereum.

The IPFS community has been thinking about immutable package managers for a long time. IPFS itself began as an immutable package manager, and it is built to make writing them much easier. @diasdavid has a project called registry-mirror , which allows you to run an npm registry locally that is backed by packages retrieved from IPFS instead of NPM directly. He’s written about a presentation he gave for it, here; the source code is here.

On a similar note, gx, a package manager for Go made by @whyrusleeping, was also mentioned in a lot of the discussions about npm and package managers, especially on Hacker News. In the past two weeks, the project went from 50 to 1000 stars, so people are clearly interested in this now.

The discussion about how to best use IPFS as a package manager is ongoing. Jump on GitHub if you have something to say; we’re listening in the FAQ and in the notes repo.

DNS outage

We’re using DigitalOcean to provide ipfs.io DNS. On Tuesday, March 24th, DigitalOcean DNS was hit by a severe outage lasting hours, which took the public gateway at ipfs.io down. We switched to DNSimple in an ad-hoc fashion and brought ipfs.io back while DigitalOcean was still down, but this incident obviously hit us on the wrong foot a bit. We’ll be working to never get taken down this way again. It’s HARD not to depend on any single points of failure. Here’s a few things we’ll do:

codify DNS zones, and tools to upload them to DNS providers

keep one or two backup DNS providers

update our monitoring and failover procedures

We’ll post a more detailed post-mortem on our blog in the next few days.

Captain.log

Aye, you might want to check the new js-ipfs Captain.log entry, matey!

Following js-ipfs roadmap, we’re close™ to having a workable js-ipfs version that can be used in the browser and in Node.js. This will mark a very important milestone on the IPFS project and enable a whole set of new distributed web applications to be possible. If you want to be part of this effort, check out our Captain.log entry to get a full update and a list of tasks you can contribute to.

@haadcode has been working on improvements to orbit-db, ipfs-log and Orbit. The message history fetching is now more stable and the UI feedback for loading messages is fixed. All this work will improve the user experience of Orbit.

js-ipfs-init

js-ipfs init works! @noffle finished the remaining pieces this week, including CLI usage. This included a handful of auxiliary PRs that cascaded out of that work. This makes the js-ipfs init process produce IPFS repos that are compatible with go-ipfs’.

Dictionary support for the zlib JavaScript Implementation, pako

One of the significant contributions made this week was the addition of ‘dictionary’ support for zlib JavaScript implementation, pako . With this contribution, we are able to have a complete implementation of SPDY 3.1’s framing layer running in the browser, the default stream muxing library used in IPFS. You can find more about this contribution in the following issue and PR discussions:

@whyrusleeping wrote a tool to move content from 0.4.0 to 0.3.11 (see levart-emit). He also discovered a file descriptor leak bug in utp causing connectivity issues, and began work on datastore performance improvements.

jsipfs object cli and http-api endpoints are complete

Now you can use jsipfs object in the same way you would use ipfs object . Big thanks to Francisco Dias for leading the last miles of this goal. The complete track of the development can be found at github.com/ipfs/js-ipfs/issues/58.

Nginx metrics

The infrastructure metrics dashboard didn’t previously have HTTP request/response metrics from nginx’s point of view, but only from IPFS’s and multireq’s point of view. (Multireq is our v04x/v03x multiplexing proxy). Nginx itself provides finegrained metrics only through their commercial subscriptions. We’re now using mtail to parse metrics from nginx access logs and expose them to Prometheus. @lgierth will also contribute the nginx.mtail program upstream with mtail.

Community

Upcoming talks

On April 20th, IPFS will host a joint meetup with ConsenSys at MIT in Cambridge, Massachusetts. Sign up here!

First IPFS Meeting in NYC

We had our first IPFS meetup in New York! It went fantastically; expect an upcoming post on the Blog soon.

Meeting with NYC Mesh

@jbenet and @lgierth met with the fine folks of nycmesh.net. For the past two years they’ve been building a community Wifi network in New York City. We had lots of great conversation about wireless mesh networking and IPFS. If you live in NYC, you should come attend their meetups!

Last Monday members of the IPFS comminuty attended a blockchain workshop event held by COALA, “a collaboration between academics, lawyers, technologists and entrepreneurs who have been driving research, policy and infrastructure-building in the blockchain ecosystem for the past three years” at the New York University School of Business. @diasdavid @haad @noffle and @nginnever were in attendance as @jbenet was a part of a protocols panel, speaking on scalability and the future of blockchain technology. A recording of the event should be available on youtube in the future here.

Lisbon Research and Development Meetup

The IPFS Lisbon community had their second “Research & Development Meetup”, hosted by Uniplaces (https://www.uniplaces.com). The focus was “The Distributed Web” (Slides) and “Machine Learning + Artificial Intelligence for Recommender Algorithms”, with talks by David Dias and João Ascensão, respectively. If you are around Lisbon, make sure to join http://www.meetup.com/ipfs-lisbon-meetup to get notified about the next one. Resources for this talk can be found here.

Seattle

@whyrusleeping gave a talk introducing IPFS at ta3m seattle - Techno-Activism 3rd Mondays. Video links to come when they are posted.

BitCoin News

BitCoin news had a discussion on using IPFS and Bitcoin for Decentralised Citizen Journalism. Check it out!

BitDevsNYC

Christian Lundkvist gave a talk on IPFS at BitDevsNYC. Christian works closely with IPFS at ConsenSys.

IPFS Meme of the Week

From https://twitter.com/jplur_/status/712670265919086594. Thanks, jplur_!

Content-Type of the Week

Now that we’re using mtail to make better sense of nginx serving the IPFS-to-HTTP gateway, we can graph the frequency of content types served. We’ll showcase interesting content types served from the gateway in the coming weeklies.

The first Content-Type of the week is: application/x-chdr, which signifies a C source header file.

Contributors

Across the entire IPFS GitHub organization, the following people have committed code, created issues, or made a comment on GitHub between March 21st (noon, GMT) and March 28th. We’re autogenerating this list using this tool and this other tool, so please let us know if your name isn’t here.

This newsletter is also a community effort. If you have cool things to share for the next weekly, drop a comment about it in the next weekly sprint issue! The more people mention items they want to see in the weekly there, the easier it is to make this and send it out.

Thanks, and see you next week!

Richard Littauer

Submit feedback about this issue here, or send us feedback about the IPFS Weekly in general.