Important to note that FastCDC is about an order of magnitude for block deduplication and is generally considered the state of art for such an approach (speed of computing the hash is more important than absolutely optimal distribution of hashes).

Zero observability and alerting too. Seems like they’re planning to be a productive future member of that team.

Who? The infrastructure team that did the move didn't even tell anyone. They were decommissioning old servers, and moving the VMs to new hardware. I'm just a lowly developer that had to troubleshoot why SMSes stopped going out.

Observability and alerting is pretty standard devops. Both the dev and infrastructure teams dropped the ball here. But at least as part of remediating this you added alerting to make sure you’d notice when your twillio connection fails in the future, right?

Even better, infrastructure enabled IPv6, and the issue was closed.

In corporate software development, we work the tickets assigned, and keep our KPIs up so that we don't face the wrath of the bean counters.

This literal example is actually addressed by the Debian example - the security team has powers to shuttle critical CVEs through but it’s a manual review process.

There’s a bunch of other improvements they call out like automated scanners before distribution and exactly what changed between two distributed versions.

The only oversight I think in the proposal is staggered distributions so that projects declare a UUID and the distribution queue progressively makes it available rather than all or nothing

> The only oversight I think in the proposal is staggered distributions so that projects declare a UUID and the distribution queue progressively makes it available rather than all or nothing

That is indeed an oversight - I wish I had thought of that idea!

No worries. Feel free to popularize it. I’m more worried about supply chain security than credit :).

Also rather than a UUID a hash of the package name is probably sufficient for back compat and avoiding people trying to rotate UUIDs to get sooner / later distribution.

But the whole point of using pypi and npm is because distributions are a thing that only old graybeard boomers use.

I don’t think we read the same thing.

> Models can be pulled along other axes, however, such as whether memory locations must be tagged in order to be used in a transaction or not, etc. Haskell requires this tagging (via TVars) so that side-effects are evident in the type system as with any other kind of monad. We quickly settled on unbounded transactions.

Snip

> In hindsight, this was a critical decision that had far-reaching implications. And to be honest, I now frequently doubt that it was the right call. We had our hearts in the right places, and the entire industry was trekking down the same path at the same time (with the notable exception of Haskell)

So basically not that TM isn’t workable, but unbounded TM is likely a fool’s errand but Haskell’s is bounded TM that requires explicit annotation of memory that will participate in atomicity.

Having worked a bit on a hobby STM in C++ (spun out of a DB startup) I would have to agree. Fully transparent STM that depends on a "sufficiently smart compiler" for an imperative language with unrestricted side effects is hopeless. But I do think that a much humbler version of STM is feasible for C++ or Rust, requiring much more explicit cooperation from the programmer. I haven't worked on this for 3 years but hope to revisit it someday.

Haskell still needs TVar and it’s not an imperative language with unrestricted side effects. I think it’s bounded vs unbounded. Side effects make it more complicated perhaps but it sounds like even in a JIT language you could have done it.

It's possible (I've done it) in C++ but there are huge footguns that I'm still ambivalent about. I agree that the bounded/unbounded distinction is the key: data must explicitly be tagged as transactional. One of the benefits of bootstrapping an STM from an existing DB as I did is that this explicitness is already present.

> unbounded TM is likely a fool’s errand

It's the whole language, not just the TM code. Other languages have no way of opting out of the TM code, whereas Haskell does.

I think it is pretty sad how there are so many modern programming languages coming out that fail to actually do something novel.

The primary advantage of a new programming language is that there is no legacy code to be compatible with. The software ergonomics space has been thoroughly explored, but restricting programs to subsets with useful properties is still an untapped field. It's seen that way because professional programmers are not used to dealing with restrictions to expressive freedom. New languages are supposed to increase freedom.

The vast majority of new programming languages would benefit from the main language being as restricted as possible and then require you to opt into the features selectively.

> The primary advantage of a new programming language is that there is no legacy code to be compatible with.

Depends a lot on what you are doing. Clojure and Scala were new languages, but they had a lot of legacy code to stay compatible with. C++ falls in a similar camp.

Well, in a sense other languages opt-in to TM code by interfacing with a relational database. Similar to how in Haskell you put TVar in front of the relevant bits to opt-in.

In Haskell it's just more economic and better integrated with the rest of the language and its typesystem.

There’s no global lock. There’s a linear MutexKey<N> that a lock of Level >= N has to be acquired with. Aquiring it consumes MutexKey<N> and hands you back MutexKey<Level+1> where Level is the N of the level you’re locking.

There’s no priority inversion possible because locks can only ever be held in decreasing orders of priority - you can’t acquire a low priority lock and then a high priority lock since your remaining MutexKey won’t have the right level.

In the example it seems pretty clear to me that:

    Mutex::new(AppConfig::default());

...is meant to be acquiring a mutex protecting some global config object, yes? That's what I'm calling a "global lock".

> There’s no priority inversion possible because locks can only ever be held in decreasing orders of priority

    T1               T2
    --               --
    small_lock();
                     big_lock();
                     small_lock(); <--- Spins waiting for T1
                 

...and now any other thread that needs big_lock() spins waiting for T2 to release it, but T2 is spinning waiting for T1 to release the (presumably less critical) small lock.

If small_lock is never ever acquired without acquiring big_lock first, small_lock serves no purpose and should be deleted from the program.

Mutex::new creates a lock, it doesn’t acquire one.

Look at the API - if big_lock and small_lock are at the same level, you would need to acquire the lock simultaneously for both locks which is accomplished within the library by sorting* the locks and then acquiring. If you fail to acquire small_lock, big lock isn’t held (it’s an all or nothing situation). This exact scenario is explained in the link by the way. You can’t bypass the “acquire simultaneously” api because you only have a key for one level

Your terminology is also off. A lock around a configuration is typically called a fine grained lock unless you’re holding that lock for large swathes of program. Global as it refers to locking doesn’t refer to visibility of the lock or that it does mutual exclusion. For example, a lock on a database that only allows one thread into a hot path operation at a time is a global lock.

* sorting is done based on global construction order grabbed at construction - there’s a singleton atomic that hands out IDs for each mutex.

No, the entire point of what I was saying is that big_lock and little_lock are at two different levels.

If big lock and little lock are at different levels you won’t have a key at the appropriate level to create an inversion by trying to acquire in the first place.

T2 might “spin” waiting for small lock but assuming small lock is released at some point you’ve not got a deadlock (and by construction it’s impossible for small lock to have it’s release blocked on the acquisition of a lock that depends on big_lock).

That’s the whole point of having a level to the locks and to the key that you have to give up to acquire that lock.

Your terminology is also off. Mutexes are not implemented through spin locks. It’s an atomic operation and when lock acquisition fails you call futex_lock (or whatever your OS api is) to have the thread be put to sleep until the lock is acquired.

I think what they're trying to say is that sure it's deadlock-free but it might be sacrificing performance.

T2 sits there waiting for small_lock to be available while holding big_lock for a long time.

This bit:

> ...and now any other thread that needs big_lock() spins waiting for T2 to release it, but T2 is spinning waiting for T1 to release the (presumably less critical) small lock.

Which of course leads to conversations like can big_lock be an RWLock, ArcSwap or such.

This reply is word salad that completely fails to engage with anything I've actually said to you... please don't waste my time with more LLM generated comments.

None of it is LLM generated. You seem to fundamentally not understand how the system outlined in the blog post works and how it prevents deadlocks by construction (ie it’s impossible to write any program that deadlocks if the only mutexes used are from this library). You also seem to lack the appropriate terminology to describe what your concern is and use terminology in a way that belies either ignorance or fundamental misunderstanding of what words mean. So you lash out claiming my 100% human written comment is LLM as a way to distract from said ignorance.

I’ve tried to illuminate your ignorance for you but unfortunately I can’t do your thinking for you.

What I can do is recommend you try to write out the scenario you believe can create a deadlock and maybe then you’ll understand why it’s not possible and maybe my words will make a little bit more sense. If alternatively you succeed you can open an issue on the author’s open source library and create a blog post explaining their mistake. But until then you’re just unhappy you don’t understand and aren’t doing any being willful to remain uninformed.

Usually a global lock is a lock that is taken outside all others and is taken for large parts of the runtime (or even, everywhere the thread isn't waiting on a condition variable, file descriptor and the like).

Mutex::new(AppConfig::default()) might very well be a small, leaf mutex.

> In the example it seems pretty clear to me that:

> Mutex::new(AppConfig::default());

> ...is meant to be acquiring a mutex protecting some global config object, yes? That's what I'm calling a "global lock".

You could certainly have a global lock at the top-most level, but you're not required to. The example is just an example.

I feel like Fuschia’s DAG approach can still be made compile time lock free by either disallowing holding locks from different branches or requiring an ordering when that does happen to prevent cycles (ie you can’t acquire them independently, you have to acquire all independent branches as a single group.

If demand keeps growing (as it has been), we've got ~40-60 years of "cheap" reserves left. As helium prices start to increase, you've got price shocks down the supply chain.

There's about 40-70 billion cubic meters of economically recoverable (assuming future technology development + price increases). The complete total upper end of known geological reserves is ~60-100 billion cubic meters - that's about correct in terms of order of magnitude even if we find new deposits.

Current consumption is 180 million cubic meters/year. At a growth of 3%, you've got 80-140 years before we run out. At 5% growth it's 50-90 years.

Saying "I'm not worried about it" is true in the myopically selfish "I personally won't have to care about it". It's conceivable that your children will be dealing with it and definitely grandchildren in a very real existentially meaningful way.

It's very hard if not impossible to do predictions over century timescales. How relevant are 1926 resource problems to today? If you wrote your comment in 1926 you would be talking about rubber, fertilizer, coal, wood or oil, and 4 out of those 5 are mostly solved today.

At those timescales, mining the moon or Jupiter for helium might be realistic, so the limits of earth are no longer upper bounds.

I agree century timescales are tough, I'm not convinced 4 of 5 of your listed things have been solved.

Rubber has been replaced with oil.

Fertilizer has been replaced with Natural Gas that comes from the same place as oil.

Coal usage has been replaced/displaced primarily by natural gas, see above.

Wood, or deforestation, was a real problem in the 1920's, but many uses were replaced by plastics (oil) and natural gas. Sustainable forestry helped a ton here too once it hit the paper industry's bottom line.

Oil is certainly not solved, so we solved 4 out of 5 with the 5th.

Exactly -- that means that any analysis based on the current (as of 1926) 'reserves' or 'production capacity' for rubber/fertilizer/coal/wood would have been invalidated as soon as we switched to using oil instead. Imagine if instead of harvesting helium directly we find an economic way to split nitrogen (somehow, who knows). At that point, what you'd have to have forecasted would be the 'reserves' of nitrogen, which are functionally infinite.

That still amounts to magical thinking. And the point of the post that you’re replying to is that we didn’t actually make things better. We actually accelerated our exploitation of other resources to make up for the shortage of the others which had serious other negative side effects.

Since we’re dealing in magical hypotheticals, what if this new economical way to split nitrogen generated so much pollution that it poisoned natural water supplies? Also the “economic way” is misleading. If prices shoot up enough, then crazy things become economical like missions to other planets to retrieve it. But that’s an insane cost that has to be borne out by all humanity. Historically that worked because you increased how many people were on earth so it spread the cost out. However, it’s pretty clear the Earth is at carrying capacity for humans with our current technology which is why the population growth has slowed drastically. So increasing costs threaten to become a weight the next generation can’t lift resulting in societal collapse.

> Imagine if instead of harvesting helium directly we find an economic way to split nitrogen (somehow, who knows).

This is nonsense, from the physics point of view.

The reason why rubber/fertilizer were replaced by oil/gas products is that oil/gas has the energy needed for the (relatively simple) chemical transformations needed to obtain rubber/fertilizer from the feedstocks.

Splitting nitrogen into helium is a nuclear reaction that requires copious amounts of energy.

At least using fusion or collecting helium from Moon/Jupiter are physically sane, if economically insane.

We're definitely not mining the moon for helium, but might well end up "mining" the gas giants.

Isn’t those calculations pretty unreliable? It’s like those predictions we only have 5 or 10 years of oil left. And then we find more oil or better extraction process and we got another 10 years and so on.

Ah, "The Iranian Nuclear Bomb Deadline" ploy.

> As helium prices start to increase, you've got price shocks down the supply chain.

No shock at all if the price is relative to what's left. Shouldn't boring market pressures guarantee this, unless the government gets involved?

No, if you hit a resource limit you’ve got exponentially increasing prices for the remainder which starts to make applications not even possible anymore. It’s not a shock in terms of months, but you could easily see MRI machines skyrocket in price over a few years as helium becomes inaccessible unless someone figures out a non-helium approach to MRIs.

High(er) temperature superconductors. Not as good as liquid helium ones, but should be good enough.

That's the point, it's NOT a resource limits so there shouldn't be some exponential. As stated elsewhere, petroleum/natural gas companies are bleeding processable helium into the atmosphere, but only because there were government reserves that broke the economics. The can start capturing it.

Just in time to start extracting helium on Mars

Maybe we will build chips in space in vacuum?

> this is by design of both companies.

I’ll note that whatever other reasons it’s also the only way to make this battery efficient. Having a bunch of different TCP connections signaling events at random times is not what you want.

Ideally the app also is responsible for rendering rather than having to disclose the message but that can be challenging to accomplish for all sorts of reasons).

Wow. What a hand wave away of the intrinsic challenge of writing fault tolerant distributed systems. It only seems easy because of decades of research and tools built since Google did it, but by no means was it something you could trivially add to a project as you can today.

> fault tolerant distributed systems

I mean there were mainframes which could be described as that. IBM just fixed it in hardware instead of software so its not like it was an unknown field.

Even if that were actually true (it’s not in important ways) Google showed you could do this cheaply in software instead of expensive in hardware.

You’re still hand waving away things like inventing a way to make map/reduce fault tolerant and automatic partitioning of data and automatic scheduling which didn’t exist before and made map/reduce accessible - mainframes weren’t doing this.

They pioneered how you durably store data on a bunch of commodity hardware through GFS - others were not doing this. And they showed how to do distributed systems at a scale not seen before because the field had bottlenecked on however big you could make a mainframe.

Given the accused breaking of ceasefire shortly after agreement, not sure how this bet really gets paid out.

There were people who bet against… if there’s no one on the other side to take the opposite bet, you don’t have a bet. And you won’t get a payout.

I think the question was: Who gets the payout? The bet is: There will be a ceasefire. There was a ceasefire, but it was allegedly broken almost immediately after. So does that count as ceasefire or not? There are arguments for both sides, so you could also say it's a tie and neither party gets the cut and the bets will be refunded.

You can just read the rules for that particular prediction on Polymarket yourself. In this case according to the rules it just comes down to whether or not there is an official ceasefire by a particular time. It does not say anything about what happens if the ceasefire is broken after that time so in that case it doesn't actually matter that it was broken.

Usually the market have rules that define the bet more concretely than just "there will be a ceasefire", and resolutions can be disputed where they will be arbitrated by the market operator. For this particular market you can see that here (https://polymarket.com/event/us-x-iran-ceasefire-by), but I'll paste the current text too:

    This market will resolve to “Yes” if there is an official ceasefire agreement, defined as a publicly announced and mutually agreed halt in direct military engagement, between the United States and Iran by the listed date, 11:59 PM ET.

    For the purposes of this market, an “official ceasefire agreement” requires clear public confirmation from both the United States government and the government of Iran that they have agreed to halt military hostilities against one another, or for an official ceasefire agreement to be otherwise confirmed to have been reached by an overwhelming consensus of media reporting.

    If the agreement is officially reached before the resolution date, this market will resolve to “Yes,” regardless of whether the ceasefire officially takes effect after the resolution date.

    Any form of informal understanding, backchannel communication, de-escalation without an announced agreement, or unilateral pause in hostilities will not be considered an official ceasefire. Humanitarian pauses, limited operational pauses, or temporary tactical stand-downs will not count toward the resolution of this market.

    A broader peace deal, normalization agreement, or political framework will qualify only if it includes a publicly announced and mutually agreed halt in military engagement between the United States and Iran, effective on a specified date, or otherwise confirmed by an overwhelming consensus of credible reporting. Agreements that outline future negotiations or de-escalation measures without an explicit, dated commitment to stop fighting will not qualify.

    This market’s resolution will be based on official statements from the United States government and the government of Iran. However, an overwhelming consensus of credible media reporting confirming that an official ceasefire agreement has been reached will suffice.

So the real answer is, "whoever the market operator chooses".

This seems to involve there being an agreement in place, but not necessarily in effect or even followed, so the possibility of it having already been violated seems to be irrelevant

They put a lot more thought into the terms than the Trump administration did, that's for sure.

Still: does the combination of a deranged Truth Social posting, an obviously-pasted tweet from the Pakistani government, and a 10-point list of debatable provenance count as "clear public confirmation?"

I guess so, sort of, maybe? Fortunately I don't have a 7-figure wager at stake.

and there are always fool in market.