Mark Friedenbach

Freicoin Developer
  • Content count

  • Joined

  • Last visited

  • Days Won


Mark Friedenbach last won the day on February 26

Mark Friedenbach had the most liked content!

1 Follower

About Mark Friedenbach

  • Rank

Profile Information

  • Gender
    Not Telling

Recent Profile Visitors

616 profile views
  1. Draft - Freirepublic

    Stellar would be an example of such a protocol. However see this paper from IBM's research division about the failings of such proposals:
  2. Draft - Freirepublic

    Arcurus, if you want to advance the idea of freirepublic / republicoin into production, I would suggest finding a way to fund research into self-organized byzantine fault tolerant consensus algorithms with rigorous formal proofs. This is the missing building block that would be required in order to have UASF-mandated use for subsidy towards specific causes in freicoin. To be clear, this is not me asking for money. I'm actually not a BFT researcher and am overworked as it is. Rather it is a necessary technology that must exist for this idea to be safely deployed, and something which (despite a lot of similar-sounding noise) isn't really being researched right now by credible academics. I would very much like for there to be research into it, as using the freicoin subsidy for other purposes fixes a lot of long-term incentive problems.
  3. > Is there still a need to have a refheight in the Freicoin transaction if so why? There's a couple of ways I can answer this question. First, there is always a need for refheight in Freicoin transactions just by definition: it would be a hard-forking consensus change to remove them. This is the technically correct answer but mention it only for completeness as I assume it's not what you meant. This doesn't affect the need for reference heights in any fundamental way. Reference heights address a problem that generally applies to demurrage in bitcoin-like block chains: the demurrage rate reduces the inputs of a transaction without affecting the outputs, so if naively evaluated at block height it would introduce a back-door transaction expiry as at some point (which can be calculated) the transaction will have less input than output, resulting in a negative fee and consensus violation if it were then included in a block. There are basically three approaches to fixing this: (1) allow negative fees so long as the block as a whole doesn't have negative fees in aggregate; (2) assign a specific height to a transaction, less than the current block height, and use that for demurrage calculations so it is independent of block height; or (3) forget demurrage in the protocol and have an exponentially inflating currency instead. The first approach didn't work in 2012 as child-pays-for-parent wasn't wasn't well supported, and still isn't to some degree. The third approach we rejected for experimental reasons, although it would have had the nice property of pulling demurrage calculations out of consensus code and block validation logic. The second approach is what we did and what we're stuck with, at least on the present and future soft-fork compatible incarnations of the Freicoin block chain. Reaching more into an aside, I will say that it was never necessary that Freicoin transactions have an explicit reference height field at all, which is what breaks compatibility with bitcoin tools. The semantics of the reference height are that it is a block number which must be greater than or equal to each of its inputs reference heights, and less than or equal to the current block height. This happens to be a more strict subset of the rules for transaction lock-time! So an extra refheight isn't really necessary and never was--we could have just double-purposed the nLockTime field to be a reference height, making its use mandatory and adding extra constraints and restrictions. But alas we didn't notice this until years after the initial release of Freicoin, and that ship has sailed leaving us stuck with reference heights as an explicit extra field. Oops. > Is the demurrage calculated on the uxto set, or on the transactions itself? Reference height of inputs are stored in the UTXO set; demurrage is calculated during transaction validation. In bitcoin there is a check that sum(inputs) >= sum(outputs) in any context where a transaction needs to be validated -- e.g. when adding to your mempool or when encountered in a block. In freicoin we do the exact same thing except in the summation of inputs we adjust each term by the demurrage rate calculated from the difference of the transaction's reference height from its input's reference height. > If on the transaction itself is then pruning of old transactions possible in Freicoin? Yes, there has never been any restrictions on pruning old blocks. There just hasn't been an official release that supports this feature, which was added in bitcoin v0.11.0.
  4. A little bit of a technical update I want to share. As part of the refactoring and rebasing effort I've worked out a way to eliminate Freicoin's extra dependencies while at the same time achieving a 10x performance boost for the mathematical calculations we use these libraries for. I'm quite excited about this, although for reasons that might be obscure for those without a technical background. If you don't understand this post, that's okay. The TL;DR is that we can make Freicoin faster while further aligning the code base with bitcoin to minimize the maintenance burden. Freicoin uses the GNU MP and MPFR multi-precision arithmetic libraries for performing cross-platform portable demurrage calculations within consensus code. Calculating demurrage factors involves taking the per-block demurrage rate r = (2-20 - 1)/(2-20) and raising it to the nth power, where n is the relative depth of the output (next block height - output's refheight). The fastest way to do this on workstation or server-class hardware is to use the CPU's floating point unit to perform the exponentiation. Unfortunately there is just absolutely no way that this can be done safely and consistently inside of consensus code. CPU manufacturers do not guarantee the bit-for-bit accuracy of operations compared with other CPUs of the same target architecture, much less with reference values or cross-platform targets. There was at one time a dream of decimal floating point (IEEE 754-2008) which would have had reproducibility as a requirement, but alas that never took off. The solution we adopted in 2012 was to use GNU MP (a multi-precision exact big integer and rational arithmetic library) and GNU MPFR (a multi-precision software floating-point library with rigidly defined semantics) to implement precise, cross-platform software floating-point demurrage accounting. This has the advantage of getting the job done, but there are some downsides: Performance. Freicoin suffers from this use of external multi-precision arithmetic libraries. Although optimized for speed GNU MP simply cannot approach the raw performance of directly adding two 64-bit numbers in hardware, and the multi-precision heap allocations add overhead and indirect storage causing latency. MPFR on the other hand is designed for precise, cross-platform behavior in simulations, not frequent high performance calculations. It is also overkill given that we use the library for one single calculation only, for which a generic library like MPFR would not provide support for specialized optimizations. Reliability. Both GNU MP and MPFR use dynamic (heap allocated) memory for their bignum representations. That means that there potentially is allocation or reallocation happening every single time a call is made to GNU MP or MPFR arithmetic functions, and potentially an out of memory exception is generated. Originally this was every single time any kind of arithmetic was done on freicoin values. With the recent input-truncation soft-fork it is already possible to reduce this to just demurrage calculations. However that still means that there is the possibility for abnormal or transient exceptions to occur in code that in bitcoin consists of normal arithmetic operations. This code was not audited for failure modes as a result of this different behavior, and there is added risk of local node instability as a result. Maintenance. Bitcoin has special scripts to make reproducible builds of all of its dependencies. These scripts are custom developed and maintained by a sizable Bitcoin Core development team, and these build scripts are very frequently updated as bitcoin matures. These scripts would have to be modified to include reproducible builds of GNU MP and MPFR, and those changes maintained and re-integrated with each new release of bitcoin. This is a bit of a sore point for me because I originally fell off the wagon of providing regular maintenance updates to Freicoin on release 0.9, in which the build system changed quite drastically due to the switch to autotools. Legal. GNU MP and GNU MPFR are both distributed under the terms of the LGPL. Because official Freicoin binary distributions are statically linked, that meant that they are also GPL programs. Now I'm a strong advocate of libre software, user rights, and the GPL, so on the face of it this was not too concerning to me back in 2012. However the Freicoin software repository says like bitcoin that it is MIT/X11 software licensed, and these sort of viral software licensing traps are difficult to see, so I can see how someone might run into GPL license violations as a result, without intent of doing so. There is a permissively licensed fork / re-implementation of GNU MP that we could use, but not alternative for MPFR. Given that the rebase to bitcoin master (0.16 as of now) would require re-doing the pieces which use GNU MP and MPFR anyway, it seemed like an opportune time to look at replacing these dependencies with something faster, safer, in-house, and permissively licensed. Thanks to input truncation, all the regular accounting can be reverted back to 64-bit integer arithmetic instead of multi-precision rational numbers. The trickier part is the demurrage calculation, which still needs to do an internal real-number precise exponentiation. To remove the dependency on GNU MPFR I wrote an integer-math-only exponentiation of the freicoin demurrage rate using pre-calculated look-up tables, and through exhaustive testing proved it to be bit-for-bit identical in its results compared with the GNU MPFR version. I then went through and applied as many approximations and reductions of internal precision as I could without generating incorrect results, to get the absolute fastest implementation. Here's some raw numbers: The first column is the algorithm name. The "mpfr_mul" is the algorithm currently used in freicoin, and "mpfr_div" and "mpfr_small" are variations upon it. The "small" variation reduces internal precision to as low as can be done without errors in exhaustive testing, which would be a cheap one-line change to the existing method. The "apu" algorithm is the new integer-math-only implementation using pre-generated lookup tables for the exponentiation. The "f32"/"f64"/"f80" algorithms use single-precision (float), double precision, and extended precision (long double) floating-point math on the CPU, as a point of comparison for timing. The "div" variations calculate the inverse of the demurrage rate, which is potentially a faster exponentiation to perform albeit at the price of a slow division at the end. There is an "apu_div" method, but it is unfinished so I did not include numbers for it. The second column gives the average run time per evaluation of the demurrage calculation on my laptop. Neither the "mpfr" nor "apu" methods have constant time performance -- the runtime speed should have some logarithmic dependency on the number of set bits in the integer representation of the relative depth (the exponent in the demurrage calculation)--a point I'll get back to in a minute. For the "mpfr" family this effect is small, but for the "apu" family it is more pronounced and the worst case performance should be about twice what is reported here. But conversely the expected execution time depends on the average age of UTXOs, which would be way less than the exhaustive testing reported here. The hardware accelerated floating point operations should be constant-time on most CPUs. The numbers above were calculated based on an exhaustive test, for which the "average" age of a UTXO is about 1,000 years. Here are more realistic numbers with an average age of 1 year: Note that mpfr_small and apu_small are 22% and 30% faster, respectfully. (Also note that apu_small is about as fast as performing the exponentiation on actual floating point hardware! That was a nice surprise.) The final column gives a triplet of numbers reporting (1) the number of errors found in exhaustive testing in which the demurrage was greater; (2) the number of test cases which matched the reference implementation; and (3) the number of errors found in which the demurrage was less than the reference implementation (mpfr_mul). Note the intrinsic errors involved with using hardware floating point, even for "f64" and "f80" which unlike "f32" should have had sufficient internal precision to perform this calculation exactly. Hardware floating-point is not to be trusted. The clear winning approach is "apu_small", and I will be replacing the GNU GMPR demurrage calculation code with this technique, using an implementation that has been exhaustively tested to be 1:1 identical in results with the multi-precision implementation that is presently in the Freicoin consensus code. What I did not expect going into this is that this implementation is not just as good as MPFR's general approach, but is in fact nearly 20x more efficient in common use cases! And I am blown away that it is nearly as fast as computing demurrage directly in hardware floating point. While the difference is unlikely to have profound effects outside of a few special cases (e.g. calling 'getbalance" on a very large wallet), it will nevertheless reduce latency of various transaction processing and UI code, thereby delivering a better user experience. Far more critically it will improve the safety and security of the Freicoin reference implementation while significantly reducing the work that goes into keeping Freicoin up to date with upstream bitcoin. That's exciting to me at least You can expect a new version of freicoind with these improvements integrated to be out soon, as that is what I am currently working on.
  5. Development update

    Merry Christmas everyone! It's been a while since I posted here. My apologies. I've been quite busy getting a proposal for MAST implemented and reviewed, which is exciting in its own right [0,1]. I now have a bit more free time to devote to freicoin in the coming months. Back in September we agreed to wind down the foundation and to burn the left over funds after finishing the remaining charitable matches. I pulled out the donation matching code and started updating it to work with the current versions of its dependencies. However there was one snag that caught me by surprise: the script expects the foundation keys to be loaded into the wallet of the node it uses, meaning that testing of the script requires a hot wallet of the foundation funds. I'd forgotten that watch-wallet support wasn't added to bitcoind until 0.10. So for safety reasons, as leaking this key material would be devastating, I'm working on the freicoind rebase right now instead. Rebasing to higher versions of the upstream bitcoind is made significantly easier in theory by the prior per-input value truncation soft-fork we deployed in the last release. In practice however we need to do a big chunk of work to refactor the freicoin patches to make use the simpler approach this allows, and then we will see the gains. I'm working on that right now, switching the code for the latest official release to use input truncation and taking advantage of the patch size reduction that allows before rebasing to more recent versions of bitcoind. I am also getting the upstream gitian build process working for freicoin rather than the custom solution I had been using in prior releases. I'll make a new post when these refactored and rebased releases are ready for testing and review. Also I have one more announcement: there will not be official support (from me) of Freicoin-Qt going forward, and it will not be included in the official release binaries. I'm sorry, but maintenance of GUI wallet takes a lot of time and I have limited resources to devote. Hopefully someone else is able to take up this burden and we can re-add support for it. But even if no one does, the reference wallet remains accessible from the command line and RPC, which is what services need, and consumer wallets are maintained by the Freicoin Alliance (Android and Electrum I believe?). One of the reasons for this is that I want to devote time to support hardware wallets, using the Ledger Nano S platform, so there is that benefit That project is waiting until the rebase to current bitcoin core is complete however. Onwards and upwards! [0] "Merkle branch verification & tail-call semantics for generalized MAST" [1] "An explanation and justification of the tail-call and MBV approach to MAST"
  6. The end of the foundation

    The last few posts got sidetracked onto a separate unresolved issue, but I think the core question of this thread is resolved: the delayed coin matching will be performed, completing our obligations for 1:1 donation matching, and then the remaining funds will be provably destroyed. The two blockers on this are updating the coin matching scripts, since code atrophy means they no longer run on recent versions of linux. The bigger problem is making sure that the funds being sent out are sent to wallets people still hold the keys to and haven't been compromised. I'll be reaching out to the contact points for the larger donations to make sure that those funds aren't being black holed, or worse. The problem with volunteer effort is that it comes in fits and spurts though. The past month I've had ~zero disposable free time due to both our annual company meeting and a drive to get a new feature to crypto currency in general: This approach to MAST and the opportunity to deploy it quickly presented itself rather unexpectedly, and most of my free time the last month was spent on that. We've hit a milestone though and work on it won't be as crazy going forward, meaning I have disposable volunteer time again. I'll get a dry run of the matching script running, and then reach out to the largest recipients to get signed messages proving key ownership. I can probably process them individually however, and destroy the funds which won't be needed, so the process isn't bottlenecked on a few missing responses.
  7. Freicoin Resources

    Bitcoin or Freicoin? Freicoin obviously doesn't have segwit until we complete rebasing and activation. Segwit is fairly well described in the bitcoin developer documentation on, but probably the best thing to do is to switch over to an actively maintained bitcoin block explorer. I don't have a specific recommendation but I know here are several.
  8. The end of the foundation

    With a 1wp we cannot expect all coins to move to the new chain -- some wallets are lost for example, so when do you stop supporting old UTXOs? With a spinoff or hard fork it would be possible to inherit the old UTXO set, but at the cost of not being able to deprecate handling the old chain data structures. A sync from genesis requires understanding the old chain up to a certain height, then switching to code that handles the new chain. Alternatively you can dump the UTXO set and import it at the changeover height, but do you still handle old style transactions at that point? Do you flag day expire them so at some point you just drop compatibility? What date do you choose for that? None of the choices there seem obvious, clean, or desirable. The 1wp with dual subsidies is a middle ground. You allow people to voluntarily move coins over and the new chain code never has to understand the old one except as is required for the peg code. You limit the maximum transfer however to something slightly larger than the current issuance. It will take some time for subsidy to play out on the old chain enough to exhaust that peg in limit -- longer if there are substantial lost coins or if people are asleep at the wheel. During that time both chains are operational. But once it is done you can remove the pegin and legacy transaction handling code from the new chain, and the old chain dies off (because subsidy can no longer be transferred to the new chain so it loses value). As mentioned by Skaro it is relevant to this thread because it achieves something close to option 3, but in a regulatory compatible way while also incentivizing dual use of the chains during the upgrade process, and solving the old UTXO problem once and for all.
  9. The end of the foundation

    Can you expand on that Jorge?
  10. Future of Freicoin

    Thanks fedde, that's good to know! Certainly some review and testing of new releases would be helpful in allowing us to pipeline things.
  11. Future of Freicoin

    I can't agree enough that we are currently limited by developer time. We need more developers. However I'm very cautious about the ICO craze. ICOs are securities offerings, and doing or promoting an unlicensed securities offering is a serious offense punished by massive fines, jail time, and a lifetime ban of working in the industry. People haven't been burned ... yet. But it's just a matter of time. It can be frustrating to watch developers with nearly zero experience raise huge sums of money for poorly defined projects, and even more frustrating to see the biggest offenders get a pass by regulatory authorities as they are grandfathered in under the new regime. However I'd still rather do this the right way, the morally and legally justifiable way, even if that road is a little harder. The situation is not so dire though. The features I would like to get into this new freicoin are mostly already written -- it's just a matter of polish and integration.
  12. The end of the foundation

    Hrm... I spent all morning dwelling on this and I think the situation might not be as hopeless as we (Jorge and I) thought before. I am on mobile and can't type a full description, but TL;DR mimblewimble demonstrates a mechanism for non interactively joining ballots together such that the resulting aggregate vote tally doesn't reveal the amounts of the individual vote. To limit loss of privacy if the federation is compromised or corrupt, you could do a coin join vote so that your individual holding is not revealed. I'm not sure how realistic that would be though.
  13. The end of the foundation

    Arcurus, all of the approaches I've seen described by you are either centralized, or vulnerable to attack if made anonymous and therefore trust/identity based. Please correct me if this is not the case. Republicoin was made nearly-decentralized by using proof of stake as a vote. 1 kria = 1 vote. However the problem is that the vote counting is done by miners in a chain, and while the miners can't create votes, they can censor them. So the miners really control the election by disenfranchising people they don't like. Not decentralized. There is a semi-trusted solution to this where a strong federation works together to do the vote counting. That could have been a solution 5 years ago, if the technology had been invented then. But alas none of this is compatible with confidential transactions. And I'd rather have confidential transactions than a semi-trusted republicoin.
  14. The end of the foundation

    Well, 5 years later it's still not clear that a decentralized solution is even possible. And if it's not decentralized.. then what are we doing here? It'd be so much simpler to have a central ledger that manages demurrage in that case. I will say that I am solely and exclusively interested in decentralized solutions only, in both the near and long term.
  15. Future of Freicoin

    After the random value is released, I go back and make up a proposed block that's even closer.