I was recently having a conversation with someone about the new segregated witness work (terrible name!) in Bitcoin and I realised that most people aren't aware of a simple and interesting fact: any possible feature upgrade to Bitcoin could be done as a "soft fork". So it's explanation time!
First, let's remind ourselves of the difference between "soft" and "hard" forks. A "hard" fork is a change in the bitcoin software protocol which will cause older versions of the bitcoin software client not to recognise the new network as a valid one. If a hard fork occurs, an old miner running old software will permanently reject the first "hard forking" block and basically act as if all post-hard-fork software is dead to it. The blockchain then splits in half with (hopefully) the bulk of present day mining power and applications using the new software protocol and its new rules, while the old protocol and its old rules continue trickling along at a snails pace thinking it was suddenly abandoned. As you can imagine, this is not something you want to be on the wrong side of. As a result, hard forks take a lot of work and preparation, probably involving a very long wait time where everyone has a chance to upgrade their software to try and make sure no one is left behind.
By contrast, a "soft" fork is a change in the Bitcoin software protocol that will still leave older software clients happy with the new blocks/blockchain. Old miners may not be producing valid blocks after the change, but they won't see the new chain as invalid. So instead of two separate block-chains going forward after the soft fork starts, there will be just one (with maybe a slightly higher rate of orphan blocks until the old miners get properly upgraded). Soft forks have the advantage of being less difficult to deploy (because you really only need the miners on board to do it), but the disadvantage is that the only thing a soft fork is capable of doing is making the restrictions on blocks stricter. This is obvious if you think about it--loosening any restrictions on block making would risk creating blocks that old software doesn't consider valid; while just making the rules tighter guarantees that any new blocks still meet the old conditions.
In many people's eyes the soft fork looks strictly better, but it's more complicated than that (as we'll see). So many people are surprised to learn that basically any possible feature can be implemented as a soft fork. How can you add arbitrary new features just by making the block rules stricter? To see, it's best to consider a simple example.
In a normal bitcoin transaction, I prove that I own the bitcoins I'm spending by signing off on it with the private key that owns the bitcoins. But beyond a valid signature, the transaction I'm signing could take many possible forms, like choosing to include or not include an OP_RETURN entry with extra random data. The only restriction the protocol enforces is that my transaction has to be validly formatted and validly signed. The rules are the same no matter which address I use, even though all the addresses have different keys. This is because Bitcoin doesn't apply any extra rules based on the particular key that I use.
But what if it did? What if for just this one particular key, we modified the software to require something more before the transaction is accepted? For example, we might say that if you want to spend the bitcoins controlled by this address, you have to give a valid signature and also include the hash of the most recent Ethereum block in an OP_RETURN. That would be a weirdly specific condition, but as long as everyone upgrades their software to enforce this rule, no transactions that violate it would end up in the blockchain any more. In fact, miners would even reject any block that included a transaction using this key which didn't have the most recent Ethereum block hash. We could even add a "maximum withdrawal size" of .01 BTC, a limit of one withdrawal per block, put a bunch of bitcoins in the address, and then publish the key to the internet. With those rules, we would have essentially created a bounty for people to submit the Ethereum block hashes to the Bitcoin network in return for rewards of .01 BTC.
Let's take a step back. Would a change like this be considered a soft fork or a hard fork? Surprisingly, as radical a change as this would be, it's still technically a soft fork. In other words, old clients look at these new "Ethereum block hash transactions", ignore the OP_RETURN, and just check the signature. The signature checks out, so they go on their way none the wiser. The old clients don't enforce the new rules, but since the new software does, it becomes a de facto rule of the network anyways. Soft fork completed!
Sounds pretty easy, hey? Instead of the Ethereum block hash bounty we could basically make up any new rule we want, just start applying it to a particular public key's transactions, publish the private key, and away we go! Right?
Well...not exactly. Just because a rule sounds simple doesn't mean that it is. Think about what the Bitcoin software would have to include to enforce the restriction "have the most recent Ethereum block hash". First, it has to include a bunch of additional code that runs only if a certain key is detected being used. Then, to make sure only valid Ethereum block hashes are accepted, it has to include an entire Ethereum client, and nodes will have to download the entire Ethereum blockchain just to validate it and make sure no fake hashes get the .01 BTC bounty. That's a pretty heavy hit, and remember that every miner has to do this all the time, or they might accidentally create invalid blocks that break the new rules, and lose 25 BTC. Plus, this "simple soft fork" basically turned Bitcoin into a conjoined twin of Ethereum. It would nearly double the size of the Bitcoin code base! Wouldn't it be easier to just write an Ethereum smart contract that manages the bounty, rather than have every single Bitcoin client running a full Ethereum node? Most miners would probably just refuse to process the new transaction type at all, rather than go to all that trouble.
In the end, this is why not everything should be a soft fork. Soft fork isn't strictly synonymous with "easy change". In particular, making the code base messier risks introducing additional bugs and vulnerabilities, as well as hindering performance. We can soft fork in the segregated witness protocol, soft fork in a Turing Complete instruction set complete with gas, even soft fork in larger block sizes! But sometimes a hard fork is the better way to do things. It keeps the code base trim, secure, and efficient. It keeps block and transaction formats sane, and makes code which handles the blockchain easier to write. Yeah, it takes a lot of social coordination. But the result is a coherent, updated, clean, secure, performant protocol. And isn't that what we're aiming for anyway? Just my two cents.