Coming up with a title that explains the full story here was difficult, so I’m going to try to explain quickly.
Yesterday, Docker sent an email to all Docker Hub users explaining that anyone who has created an “organisation” will have their account deleted including all images, if they do not upgrade to a paid team plan. The email contained a link to a tersely written PDF (since, silently edited) which was missing many important details which caused significant anxiety and additional work for open source maintainers.
What a shitshow. We really have to start worrying about the future of Github, too, since I find it highly unlikely Microsoft isn’t planning similar moves in the future. If you’re hosting code at Github, I’d suggest looking at alternatives sooner rather than later, so you don’t end up like the people affected by something like this.
GitHub is different. You can download all of your repository and transfer it, history included, to another provider (or privately host it if you like).
So if Microsoft does pull access to Github most people should not miss a beat.
mkone,
I was going to say the same thing.
Yes, there are github specific things (like issue trackers), but the code and documentation, along with the entire history can easily be ported. In fact, every time you “git pull” locally, you are making a backup.
Speaking of docker, github allows you to host those images as well:
https://docs.github.com/en/actions/publishing-packages/publishing-docker-images
(Again they are not saints, they have shut down services in the past: https://www.guyrutenberg.com/2013/02/08/github-stops-offering-binary-downloads/)
How is it different? you can use alternative docker hubs as well. And yes your docker composer files or what ever orchestration will die a bad death if the path moves, but github will kill a ton too.
I don’t think they’re that different in terms of difficulty switching providers. But I’d have to give the blast radius and the additional tooling around GitHub to mean it would be much worse.
Errm – Sure, hosting can be moved anywhere, but where will all their Github Actions run?
GitLab still offers 400 compute minutes per month on their free plan and they “provide free Ultimate licenses, along with 50K compute minutes/month, to qualifying open source projects, educational institutions, and startups”.
BitBucket offers 50 build minutes per month on their free plan.
CircleCI offers 30,000 credits per month on their free plan, which they describe as “Up to [6,000 build minutes] per month” (with the tooltip on that number clarifying it as “when using a small Docker resource class”.)
AppVeyor offers unlimited public projects with concurrency 1 on their free plan.
…and those are just the current states of the ones I remember off the top of my head from before GitHub Actions was offered. Literally the only hosted CI service I was ever aware of that no longer offers a “free for open-source” option in the GitHub Actions era is Travis-CI.
A quick jump over to Wikipedia also reveals Semaphore, which offers “$10 free credits = 1,300 minutes” on their “Free: for private, hobby, and open-source projects” plan and they confirm that in their FAQ as “Linux and macOS open source projects get free parallel CI on Semaphore.”.
Leaving GitHub would be annoying, and you need specialized tooling to dump things like your issue tracker via their API (as opposed to your repo and wiki which are git-backed), but CI services are hardly an endangered species and many of them also support continuous deployment on their free plans.
…and, while their messaging is mixed, with some people saying that the sunsetting of travis-ci.org converted open-source support into a one-time grant of 10,000 credits, their current (as of five minutes ago) “Welcome to Travis CI” e-mail contains an “If you’re building open-source public repositories and want to signup for an open-source plan, please contact Travis CI.” message which suggests it might be more that they switched to a “for qualifying open-source projects” model rather than automatic approval for any public git repo.
ssokolow,
Maybe I missed it, but I’d don’t see anything on their website about open source and even their “hobby” starter plan looks rather expensive to me. The other competitors you listed look to be cheaper and better but docker and travis could be a sign of things to come elsewhere too. I don’t know if FOSS can count on them staying free.
It’s in the welcome e-mail they send you when you sign up for the free trial or sign onto travis-ci.com for the first time after the sunsetting of travis-ci.org.
It’s possible, but I don’t think one data point out of seven is enough to predict a trend in the world of CI hosting. You could just as easily argue that Travis-CI and Docker are anomalies and SourceForge and Savannah’s long lives are evidence that free services for open source projects are generally viable.
ssokolow,
Sure, I don’t think we need to ring the alarm bells yet either. There are still free options. If more were to follow suit it could spell a trend.
I honestly don’t have a good read on what level of commitment a company like microsoft has for FOSS on github. Obviously they bought the platform with the intention of upselling packages. Is there a point when they’ll decide it makes sense to offload the freeloaders who aren’t paying? It would produce bad publicity in FOSS projects, but would this actually push away their paying users? I don’t know.
As an update, I’d say it certainly helps to provide the stick to good PR’s carrot that Gitea (the Go-based easy-to-self-host Github clone) just released a preview release of “Gitea Actions”… which aims to become drop-in compatible with Github Actions over the course of future releases.
Kinda hard for Github to feel they’ve got room to dictate terms to big customers if there’s a self-hostable competitor that’s doing its best to be easy for people to migrate to and is a single-file binary with embedded assets, support for all major database engines including SQLite, and performance good enough to run on an ARM-based single-board computer.
Heck, they’ve actually outdone Github in some ways, in that it also provides support for self-hosting private package repositories for various package managers from the same daemon. (Cargo, Chef, and Conda were just added in the newest release.)
ssokolow,
That’s good to know. Honestly I am not keen on adapting platforms that don’t have a viable exit strategy. As such, self hosting and being able to migrate to other providers are extremely important IMHO.
However speaking to whether ms have the room to dictate new terms for github, that’s unclear to me. Ultimately their motivation is profit and control in pursuit of said profit. Obviously they dislike alternatives that chip away at MS control; clearly they stand to loose control here. As much as commercial terms would put off freeloading developers, from a corporate profit perspective it’s possible these devs cost more than they’re worth having as customers. It could make financial sense to let go of those you can’t control and to let a competitor take on unprofitable customers instead.
I know this isn’t what many FOSS users will want to hear, but they may be the least valuable customers to have. To be clear, we shouldn’t make any definitive conclusions without more data, but I do think FOSS projects ought to be prepared for things to go south in terms of a decline in free services in case it happens.
Microsoft has the ability to financially support github, and most of the open source world survives on charity server hosting. The problem remains financing as Docker seems to just be Docker. No server support contracts like Ubuntu/Red Hat, so no real revenue stream to stay around, and behaving like a private company this way means no charity server hosting like the rest of the open source world enjoys. Anything of value they have will be forked and renamed, but I guess at the end of the day they hope to somehow extract money from the fact their niche is in the corporate server space.
I believe there are many centralized technologies that could be implemented in a more publicly distributed way including open source hosting. It would be interesting to see where distributed technology would be at in a world were P2P services and companies hadn’t gotten crushed by RIAA & MPAA lawsuits during the formative years of the internet. For better or worse they used the courts to shape the internet around centralized services and from that point on virtually all the investment today goes into centralized solutions rather than distributed solutions.
P2P has never been a legal issue with open source. I’m sure I can still get Ubuntu and Fedora iso file torrents if I look around their website. The issue is that it just isn’t practical. You can’t have every user download every package in the repo and make it available 24/7. That would take terabytes of space. Even if it’s just what users have downloaded, users of software with smaller niches would have to wait until someone with a complete copy comes online and starts seeding that software. Then a lot of ISPs would call you up and want to start charging you for business use based on how much you’re uploading. If it were practical, it would already be done.
dark2,
Think “DVD Jon” when the court was sending cease and desist orders against those hosting open source code in violation of the court order. Fortunately that’s behind us now, but I’m pointing it out because the US courts don’t actually care whether something is open source or not. Hackers who move faster than the courts may defy them, but unfortunately the legal threats did scare users, businesses, and investors away.
Why would every user download every package? Most P2P networks are self-scaling and the more popular something is, the more resources and redundancy it gains.
I disagree. In a universe where users, companies, governments, etc had actually embraced it originally it would be viable and we wouldn’t even be giving it a second thought. Companies including ISPs could just as easily have embraced P2P technology as a selling point and eventually everyone would be able to take it for granted. Ironically the cost to switch to centralized services would be unthinkable. However now the opposite is true because everyone is so deeply invested in centralized products and services. I maintain it didn’t have to be this way early on, but I’ll concede that now that we’re here it would cost too much to start over from square one. Today companies have become very attached to centralized technology and I’m under no delusions that it would be easy to reverse course today.
Still though, I’d like the idea to be judged fairly in terms of what it could have become had we instead developed P2P/federated/distributed technology from the get go. There are pros and cons. Many issues I have with centralized technology are solvable under a distributed P2P model.
You are aware, I hope, that it was only with the roll-out of fiber-to-the-home that telco-based ISPs outgrew a physical limitation that imposed asymmetric bandwidth.
Here in Canada, I’m on 25Mbit down, 7Mbit up on a DSL plan that should be 25Mbit/10Mbit because my line won’t hold stable at anything higher (the DSL modem’s diagnostics show a line capacity that varies between 7.5Mbit and 9Mbit upstream, depending on conditions) and, while Bell Canada did recently trench in fiber to our house in a little subdivision in the middle of nowhere with no cable TV infrastructure, the government hasn’t included it under the legislation that forces them to share their infrastructure with third-party ISPs.
Until maybe five years ago, all we could get was 5Mbit/800Kbit and that’s with us having a Bell building on the edge of the subdivision, avoiding the need to use some kind of wireless broadband option.
ssokolow,
This may not be obvious at first, but the optimal model for P2P is also asymmetric. Consider that in the P2P model upload bandwidth gets aggregated from several sources at once and it’s rare for users to be using 100% of their internet capacity, so even slower upload bandwidth can be more than enough for P2P to max out download bandwidth. It could be fun to model this mathematically if you’d like us to get into more detail.
Ultimately all of this is irrelevant though because internet bandwidth ratios aren’t set in stone anyways. ISP technology evolves around the needs of consumers and it’s natural for bandwidth allocations to reflect actual usage. If the dominant platforms had been built with P2P, then naturally that’s what ISPs would have optimized for.
Yes, I sympathize. That’s true for large swaths of the US as well. While I’m jealous of those who have access to fiber and competition in general, I do ok with the 60mbps cable service I have. DSL is considered low end, but even so many streaming services should still work on it.
https://www.highspeedinternet.com/resources/how-internet-connection-speeds-affect-watching-hd-youtube-videos
Things can be worse. US providers including ATT have been decommissioning their DSL networks in rural areas leaving customers with no internet service at all beyond cellular or satellite. My parents faced this situation and they had to ration cellular bandwidth every month or running out of data and additional bandwidth fees
DVD Jon wrote software for the illegal purpose of circumventing copyright protection (personal views on that aside, that’s the most important fact here). When it comes to illegal activity, contracts are null and void, and therefore DVD Jon’s software is not open source, or licensable in the first place. Any license attached to it is automatically null and void.
As for the rest of your P2P argument, it honestly just shows you haven’t used P2P to any significant amount to not realize the problems you would face and are willfully ignoring them. Mainly who will make all the content available, and where will the bandwidth come from? You appear to think it will be a magic solution while ignoring the main problems I’ve already mentioned, and don’t really offer a counter argument that reflects reality. So users won’t have every file in the repo, and finding rarer programs won’t take days to download because no one has them who is online and willing to seed them? Basically your solution is tyranny of the majority, and telling users of less popular software to get lost.
dark2,
I don’t quite agree with your interpretation here, but even assuming this were all true it doesn’t really refute the impact of court rulings. Regardless of what we think, the court decides for itself whether software and activities are legal or not. Even if we can convince each other that the courts were wrong (ie what about our fair use rights, etc), that doesn’t undo the court’s influence and impact on the industry.
https://arstechnica.com/uncategorized/2004/12/4467-2/
You and I might both vehemently disagree with the merits of the lawsuits against P2P software, especially when software has legal uses. But the mere fact that users and companies were being brought to court is enough to send a message to users, developers, investors, etc. For better or worse, RIAA and MPAA succeeded in steering the industry away from P2P technology, (even including napster itself) in favor of platforms with top-down control. P2P isn’t entirely gone but after years of stick and carrot coercion, it’s no longer in mainstream use even for software where P2P would be a natural fit.
Most people don’t care and shrug it off, but IMHO centralization is responsible for a lot of the monopolization, privacy and control problems we have with technology today and I feel it’s a shame that now we’re largely stuck with centralized software & services over what could have been.
Decentralized technology gives you more freedom and control, not less.
They’re saying that P2P must never be used as more than a cache/accelerator for the popular packages (which is what BitTorrent was initially envisioned as), because, unless you have a centralized fallback, like a web seed, it’s possible for niche stuff to become unavailable or very difficult to find a provider for.
ssokowlow,
You’re criticizing P2P in terms of the limited offerings today in a world where P2P investment and integration was nearly non-existent next to centralized services. But the ideas I’m proposing are in terms of what P2P would have developed into had it evolved with 20 years of industry backing and innovation.
Don’t get me wrong, self hosting is a completely viable option, but there’s no technical reason it has to be the only option, nor is there any reason it cannot be redundant. You’d have many solutions in a P2P centric world. Providers would offer hosted solutions. Many organizations, ISPs, universities, groups of FOSS developers, etc could provide modest the resources to mirror FOSS software. Many of these organizations had already taken on the responsibility of mirroring FOSS software including large linux distros and P2P technology could have offered compelling benefits in replacing HTTP/FTP distribution. This includes greater protocol level redundancy, cheaper hosting, very good scalability properties, etc.
I know we didn’t go down that route, but I ask that we all keep an open mind as to what would have been possible down the line if we had.
@Alfman, the point is DVD Jon is a terrible example with lots of legal nuance specific to his case and not open source or P2P. The case is not far off from if I were to write software for the explicit purpose of plotting murders, then releasing as open source and claiming it was only a tool meant for writing fiction. Eventually the law and courts sees 99% of use of it is illegal, and thus contracts such as the license agreement are null and void because the purpose of the software is to commit illegal activity.
I’m not going to respond to your other points about P2P as you continue to demonstrate not comprehending the argument, and not having used P2P seriously in a long time.
dark2,
Well, obviously I disagree. Not only was it very much about open source, but it’s quite an informative case especially on the balance between freedom of speech and trade secrets.
Haha, well we could certainly talk about these kinds of hypotheticals. But I’m still inclined to push back on you calling the “license agreement null and void”. The license agreement (ie BSD/GPL/proprietary/public domain/etc) is not null and void, rather the software itself would be illegal regardless of the license agreement. Maybe you don’t care to distinguish this, but to me these are technically different legal concepts that shouldn’t be conflated.
I’m ok with no response if you want to end the discussion, but I’m a bit disappointed that you’ve opted to end the discussion with an ad homenum fallacy. Agreeing to disagree would be the nicer way to go about it
That’s the elephant in the room (bazaar?) of open source: Who’s going to host all that stuff (that has to be made available for free in order for code contributions to be encouraged)? Expect this question to become more relevant as energy becomes more expensive.
kurkosdr,
It’s way too soon to conclude that this is the end of all free FOSS friendly platforms. But questions like this about FOSS funding have always been difficult to answer.. Some projects have corporate sponsors, or have paid enterprise support or dual licenses to provide some revenue. Other projects rely on donations, however I’ve heard that this can be a pittance.
The good news is that hosting expenses are a tiny fraction of developer salaries and shouldn’t break the bank of any project that can afford paid developers. But the bad news is that some FOSS software are pet projects with all volunteer developers and a budget of $0. These are the projects I’d be most concerned about assuming free platforms dried up.
I’m not too worried about this, it’s not a problem for individuals, it’s the organisations that spoof themselves as charities, a not for profit or as some FOSS Developer Group and consume huge resources that put the services at risk to individuals. In other words, the crooks spoil the party.
In fact I’d barracking for Docker to kick the crooks off!
Hijacking mail servers, hogging bandwidth, filling storage, give them all the flick and let people get on with the creative life!
In IIoT and IoT, there are a bunch of companies charging for cloud services, that fall in this category, they are basically feeding for free!
There has been a little work distributing container images via IPFS (p2p). Seems like a neat idea to self-host publicly available images without relying on a static IP or renting a server.
https://blog.ipfs.tech/2020-02-14-improved-bitswap-for-container-distribution/
kallisti5,
That’s awesome, where did you hear about this?
I’ve been touting the technical benefits of P2P forever but it mostly falls on deaf ears with a response of “meh”, haha. But it’s interesting that netflix is researching it for backend jobs and their performance numbers are very encouraging. Unfortunately when it comes to widespread acceptance, I’m skeptical of the industry being willing & able to come together at this point, I feel we missed the boat on everyday technologies & platforms being built & optimized around P2P primitives.
Oh. On a more related note… Haiku (The open source operating system) is panic shifting away from Docker since this directly puts our infrastructure at risk (If Docker dropped our images, our infrastructure would go down as k8s nodes recycle, or pods restart) This is a great example of the kinds of organizations getting hammered by this. We have a team of 4 volunteer people as part of our Open Source Org who sysadmin things. All of our images are public. (Secrets in containers are evil)
With this change, we’re shifting to Github (ghcr.io) for now (and making sure all of our infrastructure and build scripts allow rapid shifts in registry providers)
I’m taking off my Haiku hat and now personally commenting:
The summary of the above:
* Open Source projects get most of the impact via outages and panicked changes (we had a few days notice)
* It’s now bad form to hard code docker.io anywhere.
* Docker is slowly becoming less and less relevant in the world of OCI, Podman, containerd, kubernetes, etc.
So, the overall summary. Docker’s leadership is really ham-fisting this change. Docker’s relevance was holding on by a thread, and them attempting to rapidly milk the small organizations with 3-5 users for cash is going to be their downfall as users panic shift to other services (paid or unpaid)
kallisti5,
Thanks for sharing! It sounds like the industry needs a meta solution to abstract services and provide multi-homing. Something like what “matrix” does for internet communications, but for software project hosting.
It does seem that way, but it’d be nice to get the other side of the story from insiders. Maybe the unpaid FOSS projects were legitimately not sustainable for them. There is a lot of gray between pure greed and struggling to keep things going without enough income.