Mark Friedenbach

I've noticed from my logs that people have upgraded. I don't see any blocks being mined with the soft-fork version bit being set. If anyone is running their own mining that doesn't pass through the block template's version number (many don't), be sure to set the nVersion to 0x30000000 / 805306368. Or if it is version bits aware, that is the same as signaling bit #28.

Well both are necessary--it's important to indicate whether it is the 1st, 2nd, 3rd, etc. released based off the specified bitcoin release. It is also important to indicate the sequential ordering of freicoin releases, which the old numbering scheme lacked. You can look to the last number, which will always be increasing, as the freicoin build number.
Yes, these recent releases are very much a part of the effort to switch to newer bitcoin versions. The reason for these truncation soft-forks are to minimize the size of the freicoin patch against bitcoin and the required dependencies, in order to make it easier to rebase the code against more recent bitcoin versions. Since 0.8 the bitcoin source code has changed considerably between major releases making the rebasing otherwise a quite time consuming effort.

Update: Release v0.8.6.2-5 has been posted to the freicoin website. This is a maintenance-only release that has no new features. Rather, it only takes the final step of stripping out the dependency on GNU MP and MPFR entirely, such that they are no longer needed even as build-time requirements.
Notes for v0.8.6.1-4: There’s been a new release posted to the official freicoin website: v0.8.6.1-4. This release introduces the new integer-math-only demurrage calculation code as a soft-fork change to the consensus rules — the approximations in the integer math can make the result differ from the previous multi-precision floating point implementation by as much as 1 kria per input, although this error is biased in the direction that makes it soft-fork compatible with the old code. It does mean however that after activation any non-upgraded wallet code creating zero-fee transactions under the old rules is likely to create an invalid transaction. So upgrading (or paying fee) is mandatory!
This new release includes activation logic to switch over to this new integer math code path in the consensus rules once there is supermajority support by the miners, OR at block height 229376, which is about 6 weeks away. Please be sure to upgrade by then!
You can download official freicoin binaries and source code here:
http://freico.in/download/

Update: Release v0.8.6.2-5 has been posted to the freicoin website. This is a maintenance-only release that has no new features. Rather, it only takes the final step of stripping out the dependency on GNU MP and MPFR entirely, such that they are no longer needed even as build-time requirements.
Notes for v0.8.6.1-4: There’s been a new release posted to the official freicoin website: v0.8.6.1-4. This release introduces the new integer-math-only demurrage calculation code as a soft-fork change to the consensus rules — the approximations in the integer math can make the result differ from the previous multi-precision floating point implementation by as much as 1 kria per input, although this error is biased in the direction that makes it soft-fork compatible with the old code. It does mean however that after activation any non-upgraded wallet code creating zero-fee transactions under the old rules is likely to create an invalid transaction. So upgrading (or paying fee) is mandatory!
This new release includes activation logic to switch over to this new integer math code path in the consensus rules once there is supermajority support by the miners, OR at block height 229376, which is about 6 weeks away. Please be sure to upgrade by then!
You can download official freicoin binaries and source code here:
http://freico.in/download/

Update: Release v0.8.6.2-5 has been posted to the freicoin website. This is a maintenance-only release that has no new features. Rather, it only takes the final step of stripping out the dependency on GNU MP and MPFR entirely, such that they are no longer needed even as build-time requirements.
Notes for v0.8.6.1-4: There’s been a new release posted to the official freicoin website: v0.8.6.1-4. This release introduces the new integer-math-only demurrage calculation code as a soft-fork change to the consensus rules — the approximations in the integer math can make the result differ from the previous multi-precision floating point implementation by as much as 1 kria per input, although this error is biased in the direction that makes it soft-fork compatible with the old code. It does mean however that after activation any non-upgraded wallet code creating zero-fee transactions under the old rules is likely to create an invalid transaction. So upgrading (or paying fee) is mandatory!
This new release includes activation logic to switch over to this new integer math code path in the consensus rules once there is supermajority support by the miners, OR at block height 229376, which is about 6 weeks away. Please be sure to upgrade by then!
You can download official freicoin binaries and source code here:
http://freico.in/download/

Update: Release v0.8.6.2-5 has been posted to the freicoin website. This is a maintenance-only release that has no new features. Rather, it only takes the final step of stripping out the dependency on GNU MP and MPFR entirely, such that they are no longer needed even as build-time requirements.
Notes for v0.8.6.1-4: There’s been a new release posted to the official freicoin website: v0.8.6.1-4. This release introduces the new integer-math-only demurrage calculation code as a soft-fork change to the consensus rules — the approximations in the integer math can make the result differ from the previous multi-precision floating point implementation by as much as 1 kria per input, although this error is biased in the direction that makes it soft-fork compatible with the old code. It does mean however that after activation any non-upgraded wallet code creating zero-fee transactions under the old rules is likely to create an invalid transaction. So upgrading (or paying fee) is mandatory!
This new release includes activation logic to switch over to this new integer math code path in the consensus rules once there is supermajority support by the miners, OR at block height 229376, which is about 6 weeks away. Please be sure to upgrade by then!
You can download official freicoin binaries and source code here:
http://freico.in/download/

Stellar would be an example of such a protocol. However see this paper from IBM's research division about the failings of such proposals:
https://arxiv.org/abs/1707.01873

A little bit of a technical update I want to share. As part of the refactoring and rebasing effort I've worked out a way to eliminate Freicoin's extra dependencies while at the same time achieving a 10x performance boost for the mathematical calculations we use these libraries for. I'm quite excited about this, although for reasons that might be obscure for those without a technical background. If you don't understand this post, that's okay. The TL;DR is that we can make Freicoin faster while further aligning the code base with bitcoin to minimize the maintenance burden.
Freicoin uses the GNU MP and MPFR multi-precision arithmetic libraries for performing cross-platform portable demurrage calculations within consensus code. Calculating demurrage factors involves taking the per-block demurrage rate r = (2-20 - 1)/(2-20) and raising it to the nth power, where n is the relative depth of the output (next block height - output's refheight). The fastest way to do this on workstation or server-class hardware is to use the CPU's floating point unit to perform the exponentiation. Unfortunately there is just absolutely no way that this can be done safely and consistently inside of consensus code. CPU manufacturers do not guarantee the bit-for-bit accuracy of operations compared with other CPUs of the same target architecture, much less with reference values or cross-platform targets. There was at one time a dream of decimal floating point (IEEE 754-2008) which would have had reproducibility as a requirement, but alas that never took off.
The solution we adopted in 2012 was to use GNU MP (a multi-precision exact big integer and rational arithmetic library) and GNU MPFR (a multi-precision software floating-point library with rigidly defined semantics) to implement precise, cross-platform software floating-point demurrage accounting. This has the advantage of getting the job done, but there are some downsides:
Performance. Freicoin suffers from this use of external multi-precision arithmetic libraries. Although optimized for speed GNU MP simply cannot approach the raw performance of directly adding two 64-bit numbers in hardware, and the multi-precision heap allocations add overhead and indirect storage causing latency. MPFR on the other hand is designed for precise, cross-platform behavior in simulations, not frequent high performance calculations. It is also overkill given that we use the library for one single calculation only, for which a generic library like MPFR would not provide support for specialized optimizations. Reliability. Both GNU MP and MPFR use dynamic (heap allocated) memory for their bignum representations. That means that there potentially is allocation or reallocation happening every single time a call is made to GNU MP or MPFR arithmetic functions, and potentially an out of memory exception is generated. Originally this was every single time any kind of arithmetic was done on freicoin values. With the recent input-truncation soft-fork it is already possible to reduce this to just demurrage calculations. However that still means that there is the possibility for abnormal or transient exceptions to occur in code that in bitcoin consists of normal arithmetic operations. This code was not audited for failure modes as a result of this different behavior, and there is added risk of local node instability as a result. Maintenance. Bitcoin has special scripts to make reproducible builds of all of its dependencies. These scripts are custom developed and maintained by a sizable Bitcoin Core development team, and these build scripts are very frequently updated as bitcoin matures. These scripts would have to be modified to include reproducible builds of GNU MP and MPFR, and those changes maintained and re-integrated with each new release of bitcoin. This is a bit of a sore point for me because I originally fell off the wagon of providing regular maintenance updates to Freicoin on release 0.9, in which the build system changed quite drastically due to the switch to autotools. Legal. GNU MP and GNU MPFR are both distributed under the terms of the LGPL. Because official Freicoin binary distributions are statically linked, that meant that they are also GPL programs. Now I'm a strong advocate of libre software, user rights, and the GPL, so on the face of it this was not too concerning to me back in 2012. However the Freicoin software repository says like bitcoin that it is MIT/X11 software licensed, and these sort of viral software licensing traps are difficult to see, so I can see how someone might run into GPL license violations as a result, without intent of doing so. There is a permissively licensed fork / re-implementation of GNU MP that we could use, but not alternative for MPFR. Given that the rebase to bitcoin master (0.16 as of now) would require re-doing the pieces which use GNU MP and MPFR anyway, it seemed like an opportune time to look at replacing these dependencies with something faster, safer, in-house, and permissively licensed. Thanks to input truncation, all the regular accounting can be reverted back to 64-bit integer arithmetic instead of multi-precision rational numbers. The trickier part is the demurrage calculation, which still needs to do an internal real-number precise exponentiation.
To remove the dependency on GNU MPFR I wrote an integer-math-only exponentiation of the freicoin demurrage rate using pre-calculated look-up tables, and through exhaustive testing proved it to be bit-for-bit identical in its results compared with the GNU MPFR version. I then went through and applied as many approximations and reductions of internal precision as I could without generating incorrect results, to get the absolute fastest implementation. Here's some raw numbers:
The first column is the algorithm name. The "mpfr_mul" is the algorithm currently used in freicoin, and "mpfr_div" and "mpfr_small" are variations upon it. The "small" variation reduces internal precision to as low as can be done without errors in exhaustive testing, which would be a cheap one-line change to the existing method. The "apu" algorithm is the new integer-math-only implementation using pre-generated lookup tables for the exponentiation. The "f32"/"f64"/"f80" algorithms use single-precision (float), double precision, and extended precision (long double) floating-point math on the CPU, as a point of comparison for timing. The "div" variations calculate the inverse of the demurrage rate, which is potentially a faster exponentiation to perform albeit at the price of a slow division at the end. There is an "apu_div" method, but it is unfinished so I did not include numbers for it.
The second column gives the average run time per evaluation of the demurrage calculation on my laptop. Neither the "mpfr" nor "apu" methods have constant time performance -- the runtime speed should have some logarithmic dependency on the number of set bits in the integer representation of the relative depth (the exponent in the demurrage calculation)--a point I'll get back to in a minute. For the "mpfr" family this effect is small, but for the "apu" family it is more pronounced and the worst case performance should be about twice what is reported here. But conversely the expected execution time depends on the average age of UTXOs, which would be way less than the exhaustive testing reported here. The hardware accelerated floating point operations should be constant-time on most CPUs. The numbers above were calculated based on an exhaustive test, for which the "average" age of a UTXO is about 1,000 years. Here are more realistic numbers with an average age of 1 year:
Note that mpfr_small and apu_small are 22% and 30% faster, respectfully. (Also note that apu_small is about as fast as performing the exponentiation on actual floating point hardware! That was a nice surprise.)
The final column gives a triplet of numbers reporting (1) the number of errors found in exhaustive testing in which the demurrage was greater; (2) the number of test cases which matched the reference implementation; and (3) the number of errors found in which the demurrage was less than the reference implementation (mpfr_mul). Note the intrinsic errors involved with using hardware floating point, even for "f64" and "f80" which unlike "f32" should have had sufficient internal precision to perform this calculation exactly. Hardware floating-point is not to be trusted.
The clear winning approach is "apu_small", and I will be replacing the GNU GMPR demurrage calculation code with this technique, using an implementation that has been exhaustively tested to be 1:1 identical in results with the multi-precision implementation that is presently in the Freicoin consensus code.
What I did not expect going into this is that this implementation is not just as good as MPFR's general approach, but is in fact nearly 20x more efficient in common use cases! And I am blown away that it is nearly as fast as computing demurrage directly in hardware floating point.
While the difference is unlikely to have profound effects outside of a few special cases (e.g. calling 'getbalance" on a very large wallet), it will nevertheless reduce latency of various transaction processing and UI code, thereby delivering a better user experience. Far more critically it will improve the safety and security of the Freicoin reference implementation while significantly reducing the work that goes into keeping Freicoin up to date with upstream bitcoin. That's exciting to me at least
You can expect a new version of freicoind with these improvements integrated to be out soon, as that is what I am currently working on.

A little bit of a technical update I want to share. As part of the refactoring and rebasing effort I've worked out a way to eliminate Freicoin's extra dependencies while at the same time achieving a 10x performance boost for the mathematical calculations we use these libraries for. I'm quite excited about this, although for reasons that might be obscure for those without a technical background. If you don't understand this post, that's okay. The TL;DR is that we can make Freicoin faster while further aligning the code base with bitcoin to minimize the maintenance burden.
Freicoin uses the GNU MP and MPFR multi-precision arithmetic libraries for performing cross-platform portable demurrage calculations within consensus code. Calculating demurrage factors involves taking the per-block demurrage rate r = (2-20 - 1)/(2-20) and raising it to the nth power, where n is the relative depth of the output (next block height - output's refheight). The fastest way to do this on workstation or server-class hardware is to use the CPU's floating point unit to perform the exponentiation. Unfortunately there is just absolutely no way that this can be done safely and consistently inside of consensus code. CPU manufacturers do not guarantee the bit-for-bit accuracy of operations compared with other CPUs of the same target architecture, much less with reference values or cross-platform targets. There was at one time a dream of decimal floating point (IEEE 754-2008) which would have had reproducibility as a requirement, but alas that never took off.
The solution we adopted in 2012 was to use GNU MP (a multi-precision exact big integer and rational arithmetic library) and GNU MPFR (a multi-precision software floating-point library with rigidly defined semantics) to implement precise, cross-platform software floating-point demurrage accounting. This has the advantage of getting the job done, but there are some downsides:
Performance. Freicoin suffers from this use of external multi-precision arithmetic libraries. Although optimized for speed GNU MP simply cannot approach the raw performance of directly adding two 64-bit numbers in hardware, and the multi-precision heap allocations add overhead and indirect storage causing latency. MPFR on the other hand is designed for precise, cross-platform behavior in simulations, not frequent high performance calculations. It is also overkill given that we use the library for one single calculation only, for which a generic library like MPFR would not provide support for specialized optimizations. Reliability. Both GNU MP and MPFR use dynamic (heap allocated) memory for their bignum representations. That means that there potentially is allocation or reallocation happening every single time a call is made to GNU MP or MPFR arithmetic functions, and potentially an out of memory exception is generated. Originally this was every single time any kind of arithmetic was done on freicoin values. With the recent input-truncation soft-fork it is already possible to reduce this to just demurrage calculations. However that still means that there is the possibility for abnormal or transient exceptions to occur in code that in bitcoin consists of normal arithmetic operations. This code was not audited for failure modes as a result of this different behavior, and there is added risk of local node instability as a result. Maintenance. Bitcoin has special scripts to make reproducible builds of all of its dependencies. These scripts are custom developed and maintained by a sizable Bitcoin Core development team, and these build scripts are very frequently updated as bitcoin matures. These scripts would have to be modified to include reproducible builds of GNU MP and MPFR, and those changes maintained and re-integrated with each new release of bitcoin. This is a bit of a sore point for me because I originally fell off the wagon of providing regular maintenance updates to Freicoin on release 0.9, in which the build system changed quite drastically due to the switch to autotools. Legal. GNU MP and GNU MPFR are both distributed under the terms of the LGPL. Because official Freicoin binary distributions are statically linked, that meant that they are also GPL programs. Now I'm a strong advocate of libre software, user rights, and the GPL, so on the face of it this was not too concerning to me back in 2012. However the Freicoin software repository says like bitcoin that it is MIT/X11 software licensed, and these sort of viral software licensing traps are difficult to see, so I can see how someone might run into GPL license violations as a result, without intent of doing so. There is a permissively licensed fork / re-implementation of GNU MP that we could use, but not alternative for MPFR. Given that the rebase to bitcoin master (0.16 as of now) would require re-doing the pieces which use GNU MP and MPFR anyway, it seemed like an opportune time to look at replacing these dependencies with something faster, safer, in-house, and permissively licensed. Thanks to input truncation, all the regular accounting can be reverted back to 64-bit integer arithmetic instead of multi-precision rational numbers. The trickier part is the demurrage calculation, which still needs to do an internal real-number precise exponentiation.
To remove the dependency on GNU MPFR I wrote an integer-math-only exponentiation of the freicoin demurrage rate using pre-calculated look-up tables, and through exhaustive testing proved it to be bit-for-bit identical in its results compared with the GNU MPFR version. I then went through and applied as many approximations and reductions of internal precision as I could without generating incorrect results, to get the absolute fastest implementation. Here's some raw numbers:
The first column is the algorithm name. The "mpfr_mul" is the algorithm currently used in freicoin, and "mpfr_div" and "mpfr_small" are variations upon it. The "small" variation reduces internal precision to as low as can be done without errors in exhaustive testing, which would be a cheap one-line change to the existing method. The "apu" algorithm is the new integer-math-only implementation using pre-generated lookup tables for the exponentiation. The "f32"/"f64"/"f80" algorithms use single-precision (float), double precision, and extended precision (long double) floating-point math on the CPU, as a point of comparison for timing. The "div" variations calculate the inverse of the demurrage rate, which is potentially a faster exponentiation to perform albeit at the price of a slow division at the end. There is an "apu_div" method, but it is unfinished so I did not include numbers for it.
The second column gives the average run time per evaluation of the demurrage calculation on my laptop. Neither the "mpfr" nor "apu" methods have constant time performance -- the runtime speed should have some logarithmic dependency on the number of set bits in the integer representation of the relative depth (the exponent in the demurrage calculation)--a point I'll get back to in a minute. For the "mpfr" family this effect is small, but for the "apu" family it is more pronounced and the worst case performance should be about twice what is reported here. But conversely the expected execution time depends on the average age of UTXOs, which would be way less than the exhaustive testing reported here. The hardware accelerated floating point operations should be constant-time on most CPUs. The numbers above were calculated based on an exhaustive test, for which the "average" age of a UTXO is about 1,000 years. Here are more realistic numbers with an average age of 1 year:
Note that mpfr_small and apu_small are 22% and 30% faster, respectfully. (Also note that apu_small is about as fast as performing the exponentiation on actual floating point hardware! That was a nice surprise.)
The final column gives a triplet of numbers reporting (1) the number of errors found in exhaustive testing in which the demurrage was greater; (2) the number of test cases which matched the reference implementation; and (3) the number of errors found in which the demurrage was less than the reference implementation (mpfr_mul). Note the intrinsic errors involved with using hardware floating point, even for "f64" and "f80" which unlike "f32" should have had sufficient internal precision to perform this calculation exactly. Hardware floating-point is not to be trusted.
The clear winning approach is "apu_small", and I will be replacing the GNU GMPR demurrage calculation code with this technique, using an implementation that has been exhaustively tested to be 1:1 identical in results with the multi-precision implementation that is presently in the Freicoin consensus code.
What I did not expect going into this is that this implementation is not just as good as MPFR's general approach, but is in fact nearly 20x more efficient in common use cases! And I am blown away that it is nearly as fast as computing demurrage directly in hardware floating point.
While the difference is unlikely to have profound effects outside of a few special cases (e.g. calling 'getbalance" on a very large wallet), it will nevertheless reduce latency of various transaction processing and UI code, thereby delivering a better user experience. Far more critically it will improve the safety and security of the Freicoin reference implementation while significantly reducing the work that goes into keeping Freicoin up to date with upstream bitcoin. That's exciting to me at least
You can expect a new version of freicoind with these improvements integrated to be out soon, as that is what I am currently working on.

> Is there still a need to have a refheight in the Freicoin transaction if so why?
There's a couple of ways I can answer this question.
First, there is always a need for refheight in Freicoin transactions just by definition: it would be a hard-forking consensus change to remove them. This is the technically correct answer but mention it only for completeness as I assume it's not what you meant.
This doesn't affect the need for reference heights in any fundamental way. Reference heights address a problem that generally applies to demurrage in bitcoin-like block chains: the demurrage rate reduces the inputs of a transaction without affecting the outputs, so if naively evaluated at block height it would introduce a back-door transaction expiry as at some point (which can be calculated) the transaction will have less input than output, resulting in a negative fee and consensus violation if it were then included in a block. There are basically three approaches to fixing this: (1) allow negative fees so long as the block as a whole doesn't have negative fees in aggregate; (2) assign a specific height to a transaction, less than the current block height, and use that for demurrage calculations so it is independent of block height; or (3) forget demurrage in the protocol and have an exponentially inflating currency instead. The first approach didn't work in 2012 as child-pays-for-parent wasn't wasn't well supported, and still isn't to some degree. The third approach we rejected for experimental reasons, although it would have had the nice property of pulling demurrage calculations out of consensus code and block validation logic. The second approach is what we did and what we're stuck with, at least on the present and future soft-fork compatible incarnations of the Freicoin block chain.
Reaching more into an aside, I will say that it was never necessary that Freicoin transactions have an explicit reference height field at all, which is what breaks compatibility with bitcoin tools. The semantics of the reference height are that it is a block number which must be greater than or equal to each of its inputs reference heights, and less than or equal to the current block height. This happens to be a more strict subset of the rules for transaction lock-time! So an extra refheight isn't really necessary and never was--we could have just double-purposed the nLockTime field to be a reference height, making its use mandatory and adding extra constraints and restrictions. But alas we didn't notice this until years after the initial release of Freicoin, and that ship has sailed leaving us stuck with reference heights as an explicit extra field. Oops.
> Is the demurrage calculated on the uxto set, or on the transactions itself?
Reference height of inputs are stored in the UTXO set; demurrage is calculated during transaction validation. In bitcoin there is a check that sum(inputs) >= sum(outputs) in any context where a transaction needs to be validated -- e.g. when adding to your mempool or when encountered in a block. In freicoin we do the exact same thing except in the summation of inputs we adjust each term by the demurrage rate calculated from the difference of the transaction's reference height from its input's reference height.
> If on the transaction itself is then pruning of old transactions possible in Freicoin?
Yes, there has never been any restrictions on pruning old blocks. There just hasn't been an official release that supports this feature, which was added in bitcoin v0.11.0.

> Is there still a need to have a refheight in the Freicoin transaction if so why?
There's a couple of ways I can answer this question.
First, there is always a need for refheight in Freicoin transactions just by definition: it would be a hard-forking consensus change to remove them. This is the technically correct answer but mention it only for completeness as I assume it's not what you meant.
This doesn't affect the need for reference heights in any fundamental way. Reference heights address a problem that generally applies to demurrage in bitcoin-like block chains: the demurrage rate reduces the inputs of a transaction without affecting the outputs, so if naively evaluated at block height it would introduce a back-door transaction expiry as at some point (which can be calculated) the transaction will have less input than output, resulting in a negative fee and consensus violation if it were then included in a block. There are basically three approaches to fixing this: (1) allow negative fees so long as the block as a whole doesn't have negative fees in aggregate; (2) assign a specific height to a transaction, less than the current block height, and use that for demurrage calculations so it is independent of block height; or (3) forget demurrage in the protocol and have an exponentially inflating currency instead. The first approach didn't work in 2012 as child-pays-for-parent wasn't wasn't well supported, and still isn't to some degree. The third approach we rejected for experimental reasons, although it would have had the nice property of pulling demurrage calculations out of consensus code and block validation logic. The second approach is what we did and what we're stuck with, at least on the present and future soft-fork compatible incarnations of the Freicoin block chain.
Reaching more into an aside, I will say that it was never necessary that Freicoin transactions have an explicit reference height field at all, which is what breaks compatibility with bitcoin tools. The semantics of the reference height are that it is a block number which must be greater than or equal to each of its inputs reference heights, and less than or equal to the current block height. This happens to be a more strict subset of the rules for transaction lock-time! So an extra refheight isn't really necessary and never was--we could have just double-purposed the nLockTime field to be a reference height, making its use mandatory and adding extra constraints and restrictions. But alas we didn't notice this until years after the initial release of Freicoin, and that ship has sailed leaving us stuck with reference heights as an explicit extra field. Oops.
> Is the demurrage calculated on the uxto set, or on the transactions itself?
Reference height of inputs are stored in the UTXO set; demurrage is calculated during transaction validation. In bitcoin there is a check that sum(inputs) >= sum(outputs) in any context where a transaction needs to be validated -- e.g. when adding to your mempool or when encountered in a block. In freicoin we do the exact same thing except in the summation of inputs we adjust each term by the demurrage rate calculated from the difference of the transaction's reference height from its input's reference height.
> If on the transaction itself is then pruning of old transactions possible in Freicoin?
Yes, there has never been any restrictions on pruning old blocks. There just hasn't been an official release that supports this feature, which was added in bitcoin v0.11.0.

A little bit of a technical update I want to share. As part of the refactoring and rebasing effort I've worked out a way to eliminate Freicoin's extra dependencies while at the same time achieving a 10x performance boost for the mathematical calculations we use these libraries for. I'm quite excited about this, although for reasons that might be obscure for those without a technical background. If you don't understand this post, that's okay. The TL;DR is that we can make Freicoin faster while further aligning the code base with bitcoin to minimize the maintenance burden.
Freicoin uses the GNU MP and MPFR multi-precision arithmetic libraries for performing cross-platform portable demurrage calculations within consensus code. Calculating demurrage factors involves taking the per-block demurrage rate r = (2-20 - 1)/(2-20) and raising it to the nth power, where n is the relative depth of the output (next block height - output's refheight). The fastest way to do this on workstation or server-class hardware is to use the CPU's floating point unit to perform the exponentiation. Unfortunately there is just absolutely no way that this can be done safely and consistently inside of consensus code. CPU manufacturers do not guarantee the bit-for-bit accuracy of operations compared with other CPUs of the same target architecture, much less with reference values or cross-platform targets. There was at one time a dream of decimal floating point (IEEE 754-2008) which would have had reproducibility as a requirement, but alas that never took off.
The solution we adopted in 2012 was to use GNU MP (a multi-precision exact big integer and rational arithmetic library) and GNU MPFR (a multi-precision software floating-point library with rigidly defined semantics) to implement precise, cross-platform software floating-point demurrage accounting. This has the advantage of getting the job done, but there are some downsides:
Performance. Freicoin suffers from this use of external multi-precision arithmetic libraries. Although optimized for speed GNU MP simply cannot approach the raw performance of directly adding two 64-bit numbers in hardware, and the multi-precision heap allocations add overhead and indirect storage causing latency. MPFR on the other hand is designed for precise, cross-platform behavior in simulations, not frequent high performance calculations. It is also overkill given that we use the library for one single calculation only, for which a generic library like MPFR would not provide support for specialized optimizations. Reliability. Both GNU MP and MPFR use dynamic (heap allocated) memory for their bignum representations. That means that there potentially is allocation or reallocation happening every single time a call is made to GNU MP or MPFR arithmetic functions, and potentially an out of memory exception is generated. Originally this was every single time any kind of arithmetic was done on freicoin values. With the recent input-truncation soft-fork it is already possible to reduce this to just demurrage calculations. However that still means that there is the possibility for abnormal or transient exceptions to occur in code that in bitcoin consists of normal arithmetic operations. This code was not audited for failure modes as a result of this different behavior, and there is added risk of local node instability as a result. Maintenance. Bitcoin has special scripts to make reproducible builds of all of its dependencies. These scripts are custom developed and maintained by a sizable Bitcoin Core development team, and these build scripts are very frequently updated as bitcoin matures. These scripts would have to be modified to include reproducible builds of GNU MP and MPFR, and those changes maintained and re-integrated with each new release of bitcoin. This is a bit of a sore point for me because I originally fell off the wagon of providing regular maintenance updates to Freicoin on release 0.9, in which the build system changed quite drastically due to the switch to autotools. Legal. GNU MP and GNU MPFR are both distributed under the terms of the LGPL. Because official Freicoin binary distributions are statically linked, that meant that they are also GPL programs. Now I'm a strong advocate of libre software, user rights, and the GPL, so on the face of it this was not too concerning to me back in 2012. However the Freicoin software repository says like bitcoin that it is MIT/X11 software licensed, and these sort of viral software licensing traps are difficult to see, so I can see how someone might run into GPL license violations as a result, without intent of doing so. There is a permissively licensed fork / re-implementation of GNU MP that we could use, but not alternative for MPFR. Given that the rebase to bitcoin master (0.16 as of now) would require re-doing the pieces which use GNU MP and MPFR anyway, it seemed like an opportune time to look at replacing these dependencies with something faster, safer, in-house, and permissively licensed. Thanks to input truncation, all the regular accounting can be reverted back to 64-bit integer arithmetic instead of multi-precision rational numbers. The trickier part is the demurrage calculation, which still needs to do an internal real-number precise exponentiation.
To remove the dependency on GNU MPFR I wrote an integer-math-only exponentiation of the freicoin demurrage rate using pre-calculated look-up tables, and through exhaustive testing proved it to be bit-for-bit identical in its results compared with the GNU MPFR version. I then went through and applied as many approximations and reductions of internal precision as I could without generating incorrect results, to get the absolute fastest implementation. Here's some raw numbers:
The first column is the algorithm name. The "mpfr_mul" is the algorithm currently used in freicoin, and "mpfr_div" and "mpfr_small" are variations upon it. The "small" variation reduces internal precision to as low as can be done without errors in exhaustive testing, which would be a cheap one-line change to the existing method. The "apu" algorithm is the new integer-math-only implementation using pre-generated lookup tables for the exponentiation. The "f32"/"f64"/"f80" algorithms use single-precision (float), double precision, and extended precision (long double) floating-point math on the CPU, as a point of comparison for timing. The "div" variations calculate the inverse of the demurrage rate, which is potentially a faster exponentiation to perform albeit at the price of a slow division at the end. There is an "apu_div" method, but it is unfinished so I did not include numbers for it.
The second column gives the average run time per evaluation of the demurrage calculation on my laptop. Neither the "mpfr" nor "apu" methods have constant time performance -- the runtime speed should have some logarithmic dependency on the number of set bits in the integer representation of the relative depth (the exponent in the demurrage calculation)--a point I'll get back to in a minute. For the "mpfr" family this effect is small, but for the "apu" family it is more pronounced and the worst case performance should be about twice what is reported here. But conversely the expected execution time depends on the average age of UTXOs, which would be way less than the exhaustive testing reported here. The hardware accelerated floating point operations should be constant-time on most CPUs. The numbers above were calculated based on an exhaustive test, for which the "average" age of a UTXO is about 1,000 years. Here are more realistic numbers with an average age of 1 year:
Note that mpfr_small and apu_small are 22% and 30% faster, respectfully. (Also note that apu_small is about as fast as performing the exponentiation on actual floating point hardware! That was a nice surprise.)
The final column gives a triplet of numbers reporting (1) the number of errors found in exhaustive testing in which the demurrage was greater; (2) the number of test cases which matched the reference implementation; and (3) the number of errors found in which the demurrage was less than the reference implementation (mpfr_mul). Note the intrinsic errors involved with using hardware floating point, even for "f64" and "f80" which unlike "f32" should have had sufficient internal precision to perform this calculation exactly. Hardware floating-point is not to be trusted.
The clear winning approach is "apu_small", and I will be replacing the GNU GMPR demurrage calculation code with this technique, using an implementation that has been exhaustively tested to be 1:1 identical in results with the multi-precision implementation that is presently in the Freicoin consensus code.
What I did not expect going into this is that this implementation is not just as good as MPFR's general approach, but is in fact nearly 20x more efficient in common use cases! And I am blown away that it is nearly as fast as computing demurrage directly in hardware floating point.
While the difference is unlikely to have profound effects outside of a few special cases (e.g. calling 'getbalance" on a very large wallet), it will nevertheless reduce latency of various transaction processing and UI code, thereby delivering a better user experience. Far more critically it will improve the safety and security of the Freicoin reference implementation while significantly reducing the work that goes into keeping Freicoin up to date with upstream bitcoin. That's exciting to me at least
You can expect a new version of freicoind with these improvements integrated to be out soon, as that is what I am currently working on.

> Is there still a need to have a refheight in the Freicoin transaction if so why?
There's a couple of ways I can answer this question.
First, there is always a need for refheight in Freicoin transactions just by definition: it would be a hard-forking consensus change to remove them. This is the technically correct answer but mention it only for completeness as I assume it's not what you meant.
This doesn't affect the need for reference heights in any fundamental way. Reference heights address a problem that generally applies to demurrage in bitcoin-like block chains: the demurrage rate reduces the inputs of a transaction without affecting the outputs, so if naively evaluated at block height it would introduce a back-door transaction expiry as at some point (which can be calculated) the transaction will have less input than output, resulting in a negative fee and consensus violation if it were then included in a block. There are basically three approaches to fixing this: (1) allow negative fees so long as the block as a whole doesn't have negative fees in aggregate; (2) assign a specific height to a transaction, less than the current block height, and use that for demurrage calculations so it is independent of block height; or (3) forget demurrage in the protocol and have an exponentially inflating currency instead. The first approach didn't work in 2012 as child-pays-for-parent wasn't wasn't well supported, and still isn't to some degree. The third approach we rejected for experimental reasons, although it would have had the nice property of pulling demurrage calculations out of consensus code and block validation logic. The second approach is what we did and what we're stuck with, at least on the present and future soft-fork compatible incarnations of the Freicoin block chain.
Reaching more into an aside, I will say that it was never necessary that Freicoin transactions have an explicit reference height field at all, which is what breaks compatibility with bitcoin tools. The semantics of the reference height are that it is a block number which must be greater than or equal to each of its inputs reference heights, and less than or equal to the current block height. This happens to be a more strict subset of the rules for transaction lock-time! So an extra refheight isn't really necessary and never was--we could have just double-purposed the nLockTime field to be a reference height, making its use mandatory and adding extra constraints and restrictions. But alas we didn't notice this until years after the initial release of Freicoin, and that ship has sailed leaving us stuck with reference heights as an explicit extra field. Oops.
> Is the demurrage calculated on the uxto set, or on the transactions itself?
Reference height of inputs are stored in the UTXO set; demurrage is calculated during transaction validation. In bitcoin there is a check that sum(inputs) >= sum(outputs) in any context where a transaction needs to be validated -- e.g. when adding to your mempool or when encountered in a block. In freicoin we do the exact same thing except in the summation of inputs we adjust each term by the demurrage rate calculated from the difference of the transaction's reference height from its input's reference height.
> If on the transaction itself is then pruning of old transactions possible in Freicoin?
Yes, there has never been any restrictions on pruning old blocks. There just hasn't been an official release that supports this feature, which was added in bitcoin v0.11.0.

A little bit of a technical update I want to share. As part of the refactoring and rebasing effort I've worked out a way to eliminate Freicoin's extra dependencies while at the same time achieving a 10x performance boost for the mathematical calculations we use these libraries for. I'm quite excited about this, although for reasons that might be obscure for those without a technical background. If you don't understand this post, that's okay. The TL;DR is that we can make Freicoin faster while further aligning the code base with bitcoin to minimize the maintenance burden.
Freicoin uses the GNU MP and MPFR multi-precision arithmetic libraries for performing cross-platform portable demurrage calculations within consensus code. Calculating demurrage factors involves taking the per-block demurrage rate r = (2-20 - 1)/(2-20) and raising it to the nth power, where n is the relative depth of the output (next block height - output's refheight). The fastest way to do this on workstation or server-class hardware is to use the CPU's floating point unit to perform the exponentiation. Unfortunately there is just absolutely no way that this can be done safely and consistently inside of consensus code. CPU manufacturers do not guarantee the bit-for-bit accuracy of operations compared with other CPUs of the same target architecture, much less with reference values or cross-platform targets. There was at one time a dream of decimal floating point (IEEE 754-2008) which would have had reproducibility as a requirement, but alas that never took off.
The solution we adopted in 2012 was to use GNU MP (a multi-precision exact big integer and rational arithmetic library) and GNU MPFR (a multi-precision software floating-point library with rigidly defined semantics) to implement precise, cross-platform software floating-point demurrage accounting. This has the advantage of getting the job done, but there are some downsides:
Performance. Freicoin suffers from this use of external multi-precision arithmetic libraries. Although optimized for speed GNU MP simply cannot approach the raw performance of directly adding two 64-bit numbers in hardware, and the multi-precision heap allocations add overhead and indirect storage causing latency. MPFR on the other hand is designed for precise, cross-platform behavior in simulations, not frequent high performance calculations. It is also overkill given that we use the library for one single calculation only, for which a generic library like MPFR would not provide support for specialized optimizations. Reliability. Both GNU MP and MPFR use dynamic (heap allocated) memory for their bignum representations. That means that there potentially is allocation or reallocation happening every single time a call is made to GNU MP or MPFR arithmetic functions, and potentially an out of memory exception is generated. Originally this was every single time any kind of arithmetic was done on freicoin values. With the recent input-truncation soft-fork it is already possible to reduce this to just demurrage calculations. However that still means that there is the possibility for abnormal or transient exceptions to occur in code that in bitcoin consists of normal arithmetic operations. This code was not audited for failure modes as a result of this different behavior, and there is added risk of local node instability as a result. Maintenance. Bitcoin has special scripts to make reproducible builds of all of its dependencies. These scripts are custom developed and maintained by a sizable Bitcoin Core development team, and these build scripts are very frequently updated as bitcoin matures. These scripts would have to be modified to include reproducible builds of GNU MP and MPFR, and those changes maintained and re-integrated with each new release of bitcoin. This is a bit of a sore point for me because I originally fell off the wagon of providing regular maintenance updates to Freicoin on release 0.9, in which the build system changed quite drastically due to the switch to autotools. Legal. GNU MP and GNU MPFR are both distributed under the terms of the LGPL. Because official Freicoin binary distributions are statically linked, that meant that they are also GPL programs. Now I'm a strong advocate of libre software, user rights, and the GPL, so on the face of it this was not too concerning to me back in 2012. However the Freicoin software repository says like bitcoin that it is MIT/X11 software licensed, and these sort of viral software licensing traps are difficult to see, so I can see how someone might run into GPL license violations as a result, without intent of doing so. There is a permissively licensed fork / re-implementation of GNU MP that we could use, but not alternative for MPFR. Given that the rebase to bitcoin master (0.16 as of now) would require re-doing the pieces which use GNU MP and MPFR anyway, it seemed like an opportune time to look at replacing these dependencies with something faster, safer, in-house, and permissively licensed. Thanks to input truncation, all the regular accounting can be reverted back to 64-bit integer arithmetic instead of multi-precision rational numbers. The trickier part is the demurrage calculation, which still needs to do an internal real-number precise exponentiation.
To remove the dependency on GNU MPFR I wrote an integer-math-only exponentiation of the freicoin demurrage rate using pre-calculated look-up tables, and through exhaustive testing proved it to be bit-for-bit identical in its results compared with the GNU MPFR version. I then went through and applied as many approximations and reductions of internal precision as I could without generating incorrect results, to get the absolute fastest implementation. Here's some raw numbers:
The first column is the algorithm name. The "mpfr_mul" is the algorithm currently used in freicoin, and "mpfr_div" and "mpfr_small" are variations upon it. The "small" variation reduces internal precision to as low as can be done without errors in exhaustive testing, which would be a cheap one-line change to the existing method. The "apu" algorithm is the new integer-math-only implementation using pre-generated lookup tables for the exponentiation. The "f32"/"f64"/"f80" algorithms use single-precision (float), double precision, and extended precision (long double) floating-point math on the CPU, as a point of comparison for timing. The "div" variations calculate the inverse of the demurrage rate, which is potentially a faster exponentiation to perform albeit at the price of a slow division at the end. There is an "apu_div" method, but it is unfinished so I did not include numbers for it.
The second column gives the average run time per evaluation of the demurrage calculation on my laptop. Neither the "mpfr" nor "apu" methods have constant time performance -- the runtime speed should have some logarithmic dependency on the number of set bits in the integer representation of the relative depth (the exponent in the demurrage calculation)--a point I'll get back to in a minute. For the "mpfr" family this effect is small, but for the "apu" family it is more pronounced and the worst case performance should be about twice what is reported here. But conversely the expected execution time depends on the average age of UTXOs, which would be way less than the exhaustive testing reported here. The hardware accelerated floating point operations should be constant-time on most CPUs. The numbers above were calculated based on an exhaustive test, for which the "average" age of a UTXO is about 1,000 years. Here are more realistic numbers with an average age of 1 year:
Note that mpfr_small and apu_small are 22% and 30% faster, respectfully. (Also note that apu_small is about as fast as performing the exponentiation on actual floating point hardware! That was a nice surprise.)
The final column gives a triplet of numbers reporting (1) the number of errors found in exhaustive testing in which the demurrage was greater; (2) the number of test cases which matched the reference implementation; and (3) the number of errors found in which the demurrage was less than the reference implementation (mpfr_mul). Note the intrinsic errors involved with using hardware floating point, even for "f64" and "f80" which unlike "f32" should have had sufficient internal precision to perform this calculation exactly. Hardware floating-point is not to be trusted.
The clear winning approach is "apu_small", and I will be replacing the GNU GMPR demurrage calculation code with this technique, using an implementation that has been exhaustively tested to be 1:1 identical in results with the multi-precision implementation that is presently in the Freicoin consensus code.
What I did not expect going into this is that this implementation is not just as good as MPFR's general approach, but is in fact nearly 20x more efficient in common use cases! And I am blown away that it is nearly as fast as computing demurrage directly in hardware floating point.
While the difference is unlikely to have profound effects outside of a few special cases (e.g. calling 'getbalance" on a very large wallet), it will nevertheless reduce latency of various transaction processing and UI code, thereby delivering a better user experience. Far more critically it will improve the safety and security of the Freicoin reference implementation while significantly reducing the work that goes into keeping Freicoin up to date with upstream bitcoin. That's exciting to me at least
You can expect a new version of freicoind with these improvements integrated to be out soon, as that is what I am currently working on.

> Is there still a need to have a refheight in the Freicoin transaction if so why?
There's a couple of ways I can answer this question.
First, there is always a need for refheight in Freicoin transactions just by definition: it would be a hard-forking consensus change to remove them. This is the technically correct answer but mention it only for completeness as I assume it's not what you meant.
This doesn't affect the need for reference heights in any fundamental way. Reference heights address a problem that generally applies to demurrage in bitcoin-like block chains: the demurrage rate reduces the inputs of a transaction without affecting the outputs, so if naively evaluated at block height it would introduce a back-door transaction expiry as at some point (which can be calculated) the transaction will have less input than output, resulting in a negative fee and consensus violation if it were then included in a block. There are basically three approaches to fixing this: (1) allow negative fees so long as the block as a whole doesn't have negative fees in aggregate; (2) assign a specific height to a transaction, less than the current block height, and use that for demurrage calculations so it is independent of block height; or (3) forget demurrage in the protocol and have an exponentially inflating currency instead. The first approach didn't work in 2012 as child-pays-for-parent wasn't wasn't well supported, and still isn't to some degree. The third approach we rejected for experimental reasons, although it would have had the nice property of pulling demurrage calculations out of consensus code and block validation logic. The second approach is what we did and what we're stuck with, at least on the present and future soft-fork compatible incarnations of the Freicoin block chain.
Reaching more into an aside, I will say that it was never necessary that Freicoin transactions have an explicit reference height field at all, which is what breaks compatibility with bitcoin tools. The semantics of the reference height are that it is a block number which must be greater than or equal to each of its inputs reference heights, and less than or equal to the current block height. This happens to be a more strict subset of the rules for transaction lock-time! So an extra refheight isn't really necessary and never was--we could have just double-purposed the nLockTime field to be a reference height, making its use mandatory and adding extra constraints and restrictions. But alas we didn't notice this until years after the initial release of Freicoin, and that ship has sailed leaving us stuck with reference heights as an explicit extra field. Oops.
> Is the demurrage calculated on the uxto set, or on the transactions itself?
Reference height of inputs are stored in the UTXO set; demurrage is calculated during transaction validation. In bitcoin there is a check that sum(inputs) >= sum(outputs) in any context where a transaction needs to be validated -- e.g. when adding to your mempool or when encountered in a block. In freicoin we do the exact same thing except in the summation of inputs we adjust each term by the demurrage rate calculated from the difference of the transaction's reference height from its input's reference height.
> If on the transaction itself is then pruning of old transactions possible in Freicoin?
Yes, there has never been any restrictions on pruning old blocks. There just hasn't been an official release that supports this feature, which was added in bitcoin v0.11.0.

A little bit of a technical update I want to share. As part of the refactoring and rebasing effort I've worked out a way to eliminate Freicoin's extra dependencies while at the same time achieving a 10x performance boost for the mathematical calculations we use these libraries for. I'm quite excited about this, although for reasons that might be obscure for those without a technical background. If you don't understand this post, that's okay. The TL;DR is that we can make Freicoin faster while further aligning the code base with bitcoin to minimize the maintenance burden.
Freicoin uses the GNU MP and MPFR multi-precision arithmetic libraries for performing cross-platform portable demurrage calculations within consensus code. Calculating demurrage factors involves taking the per-block demurrage rate r = (2-20 - 1)/(2-20) and raising it to the nth power, where n is the relative depth of the output (next block height - output's refheight). The fastest way to do this on workstation or server-class hardware is to use the CPU's floating point unit to perform the exponentiation. Unfortunately there is just absolutely no way that this can be done safely and consistently inside of consensus code. CPU manufacturers do not guarantee the bit-for-bit accuracy of operations compared with other CPUs of the same target architecture, much less with reference values or cross-platform targets. There was at one time a dream of decimal floating point (IEEE 754-2008) which would have had reproducibility as a requirement, but alas that never took off.
The solution we adopted in 2012 was to use GNU MP (a multi-precision exact big integer and rational arithmetic library) and GNU MPFR (a multi-precision software floating-point library with rigidly defined semantics) to implement precise, cross-platform software floating-point demurrage accounting. This has the advantage of getting the job done, but there are some downsides:
Performance. Freicoin suffers from this use of external multi-precision arithmetic libraries. Although optimized for speed GNU MP simply cannot approach the raw performance of directly adding two 64-bit numbers in hardware, and the multi-precision heap allocations add overhead and indirect storage causing latency. MPFR on the other hand is designed for precise, cross-platform behavior in simulations, not frequent high performance calculations. It is also overkill given that we use the library for one single calculation only, for which a generic library like MPFR would not provide support for specialized optimizations. Reliability. Both GNU MP and MPFR use dynamic (heap allocated) memory for their bignum representations. That means that there potentially is allocation or reallocation happening every single time a call is made to GNU MP or MPFR arithmetic functions, and potentially an out of memory exception is generated. Originally this was every single time any kind of arithmetic was done on freicoin values. With the recent input-truncation soft-fork it is already possible to reduce this to just demurrage calculations. However that still means that there is the possibility for abnormal or transient exceptions to occur in code that in bitcoin consists of normal arithmetic operations. This code was not audited for failure modes as a result of this different behavior, and there is added risk of local node instability as a result. Maintenance. Bitcoin has special scripts to make reproducible builds of all of its dependencies. These scripts are custom developed and maintained by a sizable Bitcoin Core development team, and these build scripts are very frequently updated as bitcoin matures. These scripts would have to be modified to include reproducible builds of GNU MP and MPFR, and those changes maintained and re-integrated with each new release of bitcoin. This is a bit of a sore point for me because I originally fell off the wagon of providing regular maintenance updates to Freicoin on release 0.9, in which the build system changed quite drastically due to the switch to autotools. Legal. GNU MP and GNU MPFR are both distributed under the terms of the LGPL. Because official Freicoin binary distributions are statically linked, that meant that they are also GPL programs. Now I'm a strong advocate of libre software, user rights, and the GPL, so on the face of it this was not too concerning to me back in 2012. However the Freicoin software repository says like bitcoin that it is MIT/X11 software licensed, and these sort of viral software licensing traps are difficult to see, so I can see how someone might run into GPL license violations as a result, without intent of doing so. There is a permissively licensed fork / re-implementation of GNU MP that we could use, but not alternative for MPFR. Given that the rebase to bitcoin master (0.16 as of now) would require re-doing the pieces which use GNU MP and MPFR anyway, it seemed like an opportune time to look at replacing these dependencies with something faster, safer, in-house, and permissively licensed. Thanks to input truncation, all the regular accounting can be reverted back to 64-bit integer arithmetic instead of multi-precision rational numbers. The trickier part is the demurrage calculation, which still needs to do an internal real-number precise exponentiation.
To remove the dependency on GNU MPFR I wrote an integer-math-only exponentiation of the freicoin demurrage rate using pre-calculated look-up tables, and through exhaustive testing proved it to be bit-for-bit identical in its results compared with the GNU MPFR version. I then went through and applied as many approximations and reductions of internal precision as I could without generating incorrect results, to get the absolute fastest implementation. Here's some raw numbers:
The first column is the algorithm name. The "mpfr_mul" is the algorithm currently used in freicoin, and "mpfr_div" and "mpfr_small" are variations upon it. The "small" variation reduces internal precision to as low as can be done without errors in exhaustive testing, which would be a cheap one-line change to the existing method. The "apu" algorithm is the new integer-math-only implementation using pre-generated lookup tables for the exponentiation. The "f32"/"f64"/"f80" algorithms use single-precision (float), double precision, and extended precision (long double) floating-point math on the CPU, as a point of comparison for timing. The "div" variations calculate the inverse of the demurrage rate, which is potentially a faster exponentiation to perform albeit at the price of a slow division at the end. There is an "apu_div" method, but it is unfinished so I did not include numbers for it.
The second column gives the average run time per evaluation of the demurrage calculation on my laptop. Neither the "mpfr" nor "apu" methods have constant time performance -- the runtime speed should have some logarithmic dependency on the number of set bits in the integer representation of the relative depth (the exponent in the demurrage calculation)--a point I'll get back to in a minute. For the "mpfr" family this effect is small, but for the "apu" family it is more pronounced and the worst case performance should be about twice what is reported here. But conversely the expected execution time depends on the average age of UTXOs, which would be way less than the exhaustive testing reported here. The hardware accelerated floating point operations should be constant-time on most CPUs. The numbers above were calculated based on an exhaustive test, for which the "average" age of a UTXO is about 1,000 years. Here are more realistic numbers with an average age of 1 year:
Note that mpfr_small and apu_small are 22% and 30% faster, respectfully. (Also note that apu_small is about as fast as performing the exponentiation on actual floating point hardware! That was a nice surprise.)
The final column gives a triplet of numbers reporting (1) the number of errors found in exhaustive testing in which the demurrage was greater; (2) the number of test cases which matched the reference implementation; and (3) the number of errors found in which the demurrage was less than the reference implementation (mpfr_mul). Note the intrinsic errors involved with using hardware floating point, even for "f64" and "f80" which unlike "f32" should have had sufficient internal precision to perform this calculation exactly. Hardware floating-point is not to be trusted.
The clear winning approach is "apu_small", and I will be replacing the GNU GMPR demurrage calculation code with this technique, using an implementation that has been exhaustively tested to be 1:1 identical in results with the multi-precision implementation that is presently in the Freicoin consensus code.
What I did not expect going into this is that this implementation is not just as good as MPFR's general approach, but is in fact nearly 20x more efficient in common use cases! And I am blown away that it is nearly as fast as computing demurrage directly in hardware floating point.
While the difference is unlikely to have profound effects outside of a few special cases (e.g. calling 'getbalance" on a very large wallet), it will nevertheless reduce latency of various transaction processing and UI code, thereby delivering a better user experience. Far more critically it will improve the safety and security of the Freicoin reference implementation while significantly reducing the work that goes into keeping Freicoin up to date with upstream bitcoin. That's exciting to me at least
You can expect a new version of freicoind with these improvements integrated to be out soon, as that is what I am currently working on.

Merry Christmas everyone!
It's been a while since I posted here. My apologies. I've been quite busy getting a proposal for MAST implemented and reviewed, which is exciting in its own right [0,1]. I now have a bit more free time to devote to freicoin in the coming months.
Back in September we agreed to wind down the foundation and to burn the left over funds after finishing the remaining charitable matches. I pulled out the donation matching code and started updating it to work with the current versions of its dependencies. However there was one snag that caught me by surprise: the script expects the foundation keys to be loaded into the wallet of the node it uses, meaning that testing of the script requires a hot wallet of the foundation funds. I'd forgotten that watch-wallet support wasn't added to bitcoind until 0.10. So for safety reasons, as leaking this key material would be devastating, I'm working on the freicoind rebase right now instead.
Rebasing to higher versions of the upstream bitcoind is made significantly easier in theory by the prior per-input value truncation soft-fork we deployed in the last release. In practice however we need to do a big chunk of work to refactor the freicoin patches to make use the simpler approach this allows, and then we will see the gains. I'm working on that right now, switching the code for the latest official release to use input truncation and taking advantage of the patch size reduction that allows before rebasing to more recent versions of bitcoind. I am also getting the upstream gitian build process working for freicoin rather than the custom solution I had been using in prior releases. I'll make a new post when these refactored and rebased releases are ready for testing and review.
Also I have one more announcement: there will not be official support (from me) of Freicoin-Qt going forward, and it will not be included in the official release binaries. I'm sorry, but maintenance of GUI wallet takes a lot of time and I have limited resources to devote. Hopefully someone else is able to take up this burden and we can re-add support for it. But even if no one does, the reference wallet remains accessible from the command line and RPC, which is what services need, and consumer wallets are maintained by the Freicoin Alliance (Android and Electrum I believe?). One of the reasons for this is that I want to devote time to support hardware wallets, using the Ledger Nano S platform, so there is that benefit That project is waiting until the rebase to current bitcoin core is complete however.
Onwards and upwards!
[0] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/014932.html "Merkle branch verification & tail-call semantics for generalized MAST"
[1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/015028.html "An explanation and justification of the tail-call and MBV approach to MAST"

Merry Christmas everyone!
It's been a while since I posted here. My apologies. I've been quite busy getting a proposal for MAST implemented and reviewed, which is exciting in its own right [0,1]. I now have a bit more free time to devote to freicoin in the coming months.
Back in September we agreed to wind down the foundation and to burn the left over funds after finishing the remaining charitable matches. I pulled out the donation matching code and started updating it to work with the current versions of its dependencies. However there was one snag that caught me by surprise: the script expects the foundation keys to be loaded into the wallet of the node it uses, meaning that testing of the script requires a hot wallet of the foundation funds. I'd forgotten that watch-wallet support wasn't added to bitcoind until 0.10. So for safety reasons, as leaking this key material would be devastating, I'm working on the freicoind rebase right now instead.
Rebasing to higher versions of the upstream bitcoind is made significantly easier in theory by the prior per-input value truncation soft-fork we deployed in the last release. In practice however we need to do a big chunk of work to refactor the freicoin patches to make use the simpler approach this allows, and then we will see the gains. I'm working on that right now, switching the code for the latest official release to use input truncation and taking advantage of the patch size reduction that allows before rebasing to more recent versions of bitcoind. I am also getting the upstream gitian build process working for freicoin rather than the custom solution I had been using in prior releases. I'll make a new post when these refactored and rebased releases are ready for testing and review.
Also I have one more announcement: there will not be official support (from me) of Freicoin-Qt going forward, and it will not be included in the official release binaries. I'm sorry, but maintenance of GUI wallet takes a lot of time and I have limited resources to devote. Hopefully someone else is able to take up this burden and we can re-add support for it. But even if no one does, the reference wallet remains accessible from the command line and RPC, which is what services need, and consumer wallets are maintained by the Freicoin Alliance (Android and Electrum I believe?). One of the reasons for this is that I want to devote time to support hardware wallets, using the Ledger Nano S platform, so there is that benefit That project is waiting until the rebase to current bitcoin core is complete however.
Onwards and upwards!
[0] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/014932.html "Merkle branch verification & tail-call semantics for generalized MAST"
[1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/015028.html "An explanation and justification of the tail-call and MBV approach to MAST"

Merry Christmas everyone!
It's been a while since I posted here. My apologies. I've been quite busy getting a proposal for MAST implemented and reviewed, which is exciting in its own right [0,1]. I now have a bit more free time to devote to freicoin in the coming months.
Back in September we agreed to wind down the foundation and to burn the left over funds after finishing the remaining charitable matches. I pulled out the donation matching code and started updating it to work with the current versions of its dependencies. However there was one snag that caught me by surprise: the script expects the foundation keys to be loaded into the wallet of the node it uses, meaning that testing of the script requires a hot wallet of the foundation funds. I'd forgotten that watch-wallet support wasn't added to bitcoind until 0.10. So for safety reasons, as leaking this key material would be devastating, I'm working on the freicoind rebase right now instead.
Rebasing to higher versions of the upstream bitcoind is made significantly easier in theory by the prior per-input value truncation soft-fork we deployed in the last release. In practice however we need to do a big chunk of work to refactor the freicoin patches to make use the simpler approach this allows, and then we will see the gains. I'm working on that right now, switching the code for the latest official release to use input truncation and taking advantage of the patch size reduction that allows before rebasing to more recent versions of bitcoind. I am also getting the upstream gitian build process working for freicoin rather than the custom solution I had been using in prior releases. I'll make a new post when these refactored and rebased releases are ready for testing and review.
Also I have one more announcement: there will not be official support (from me) of Freicoin-Qt going forward, and it will not be included in the official release binaries. I'm sorry, but maintenance of GUI wallet takes a lot of time and I have limited resources to devote. Hopefully someone else is able to take up this burden and we can re-add support for it. But even if no one does, the reference wallet remains accessible from the command line and RPC, which is what services need, and consumer wallets are maintained by the Freicoin Alliance (Android and Electrum I believe?). One of the reasons for this is that I want to devote time to support hardware wallets, using the Ledger Nano S platform, so there is that benefit That project is waiting until the rebase to current bitcoin core is complete however.
Onwards and upwards!
[0] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/014932.html "Merkle branch verification & tail-call semantics for generalized MAST"
[1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/015028.html "An explanation and justification of the tail-call and MBV approach to MAST"

It's time to re-start this thread. As mentioned in another thread, I've been professionally busy non-stop for some months on a couple of different projects. That is currently slated to change with my new responsibilities including the maintenance of the Elements codebase, which includes most of the new features we would like to add to Freicoin. So there's a little bit of double-dipping there. Plus since I'll be traveling less and working normal hours I'll be able to spend 8-10 hours a week, evenings and weekends, working on the basic plan outlined at the top of this thread.
So let's revisit this thread. We did back-port some consensus bug fixes to the freicoin client and issued a new release of the 0.8 node software in 2015. Also included in that release was a soft-fork that changed the rules for demurrage calculation, in a way that will potentially make the client easier to maintain. We have not made good on our promise to update the client to newer versions of upstream Bitcoin Core, and that will be my first priority. The soft-fork change to demurrage calculations means that a little bit of up-front work is required to re-write those patches, but afterwards the total maintenance cost should be reduced.
The call for testers still stands. We need to develop some sort of test plan for new releases that gives us some assurance we aren't releasing bad code. Any help with this would be much appreciated.

Now looking at the specific features I suggested adding:
This is already in bitcoin, and will be in freicoin as soon as we upgrade the reference client to a more recent upstream version.
Likewise! Freicoin will get the same segwit that bitcoin is about to lock-in.
Still want this for Freicoin, but:
In the time since I wrote this, Jorge and I along with our co-workers and a partner company implemented confidential assets, which is an extension of confidential transactions that blinds the asset being transacted as well as the amount. This was a significant amount of work, and basically occupied 12-14 months of my professional career, as well as that of my colleagues.
Part of why my enthusiasm for bringing CT to Freicoin waned after writing the above words is that as soon as Andrew Poelstra invented the necessary crypto component to make confidential assets work, it became clear that we would want to wait until this new CA was finished. Proper asset support has always been an essential part of my vision for Freicoin as it is necessary to construct other forms of decentralized money (e.g. classic ripple) as well as interesting smart contracts.
Confidential assets was released this year, and made more polished in the last few months in preparation for the BC2 conference, where the necessary features for distributed exchanges were also written. It's now in a state where we can talk about brining it into production on Freicoin.
This was Mimblewimble!
No, I'm not secretly Voldemort, nor does Voldie work at my employer as far as I'm aware. However we were working on some similar ideas for non-interactive CoinJoin using confidential transactions and one-way aggregatable signatures. Voldemort independently invented the same tricks, and to his/her credit realized that it allowed A LOT more than just non-interactive CoinJoin.
It also turns out that it doesn't have to be a hard-fork change, other than the introduction of confidential transactions itself. Mimblewimble can be implemented on Elements Alpha without any changes. However it does need some work to reconcile with confidential assets, so some basic research here might be required. It is however high priority research at my employer and I am hopeful that we can see production-ready results on a relatively short timeframe.

The only other features not mentioned here are a bunch of little improvements to bitcoin script that makes writing smart contracts much easier or allows more powerful contracts. That might be a good topic for another post. Many of these are already implemented in the elements codebase, but just need some review and testing before production use.

Finally, a word needs to be said about merged mining. In fact this deserves its own post and I will make one in time, but the short TL;DR is that Jorge and I differ on our views about this. Given what we have seen happen in the Bitcoin space with respect to mining, ASICBOOST, and segregated witness, I no longer have faith that SHA256 mining can be used to secure a high-value network. And as bad as the situation is for bitcoin, it would likely be even worse for a merged mined coin, as in practice merge mining sees higher rates of centralization. I think there is a strong argument for switching to a new proof of work entirely, one which is not ASIC resistant (that's a unicorn that doesn't exist), but rather tries to minimize the performance gap between high-end general compute ad specialized hardware. Cuckoo Cycle is an example of one such PoW algorithm. In any case, I'll post my thoughts regarding this in another thread.

I hope others are as excited about bringing these new features to Freicoin as I am. I think the applications that can be built on top will do a lot to bring utility to freicoin as a medium of exchange for a generally useful block chain platform.

The last few posts got sidetracked onto a separate unresolved issue, but I think the core question of this thread is resolved: the delayed coin matching will be performed, completing our obligations for 1:1 donation matching, and then the remaining funds will be provably destroyed. The two blockers on this are updating the coin matching scripts, since code atrophy means they no longer run on recent versions of linux. The bigger problem is making sure that the funds being sent out are sent to wallets people still hold the keys to and haven't been compromised. I'll be reaching out to the contact points for the larger donations to make sure that those funds aren't being black holed, or worse.
The problem with volunteer effort is that it comes in fits and spurts though. The past month I've had ~zero disposable free time due to both our annual company meeting and a drive to get a new feature to crypto currency in general:
https://www.coindesk.com/master-plan-better-bitcoin-smart-contracts-go-live-year/
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/014932.html
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/015028.html
This approach to MAST and the opportunity to deploy it quickly presented itself rather unexpectedly, and most of my free time the last month was spent on that. We've hit a milestone though and work on it won't be as crazy going forward, meaning I have disposable volunteer time again. I'll get a dry run of the matching script running, and then reach out to the largest recipients to get signed messages proving key ownership. I can probably process them individually however, and destroy the funds which won't be needed, so the process isn't bottlenecked on a few missing responses.

The last few posts got sidetracked onto a separate unresolved issue, but I think the core question of this thread is resolved: the delayed coin matching will be performed, completing our obligations for 1:1 donation matching, and then the remaining funds will be provably destroyed. The two blockers on this are updating the coin matching scripts, since code atrophy means they no longer run on recent versions of linux. The bigger problem is making sure that the funds being sent out are sent to wallets people still hold the keys to and haven't been compromised. I'll be reaching out to the contact points for the larger donations to make sure that those funds aren't being black holed, or worse.
The problem with volunteer effort is that it comes in fits and spurts though. The past month I've had ~zero disposable free time due to both our annual company meeting and a drive to get a new feature to crypto currency in general:
https://www.coindesk.com/master-plan-better-bitcoin-smart-contracts-go-live-year/
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/014932.html
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/015028.html
This approach to MAST and the opportunity to deploy it quickly presented itself rather unexpectedly, and most of my free time the last month was spent on that. We've hit a milestone though and work on it won't be as crazy going forward, meaning I have disposable volunteer time again. I'll get a dry run of the matching script running, and then reach out to the largest recipients to get signed messages proving key ownership. I can probably process them individually however, and destroy the funds which won't be needed, so the process isn't bottlenecked on a few missing responses.

The last few posts got sidetracked onto a separate unresolved issue, but I think the core question of this thread is resolved: the delayed coin matching will be performed, completing our obligations for 1:1 donation matching, and then the remaining funds will be provably destroyed. The two blockers on this are updating the coin matching scripts, since code atrophy means they no longer run on recent versions of linux. The bigger problem is making sure that the funds being sent out are sent to wallets people still hold the keys to and haven't been compromised. I'll be reaching out to the contact points for the larger donations to make sure that those funds aren't being black holed, or worse.
The problem with volunteer effort is that it comes in fits and spurts though. The past month I've had ~zero disposable free time due to both our annual company meeting and a drive to get a new feature to crypto currency in general:
https://www.coindesk.com/master-plan-better-bitcoin-smart-contracts-go-live-year/
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/014932.html
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/015028.html
This approach to MAST and the opportunity to deploy it quickly presented itself rather unexpectedly, and most of my free time the last month was spent on that. We've hit a milestone though and work on it won't be as crazy going forward, meaning I have disposable volunteer time again. I'll get a dry run of the matching script running, and then reach out to the largest recipients to get signed messages proving key ownership. I can probably process them individually however, and destroy the funds which won't be needed, so the process isn't bottlenecked on a few missing responses.

Bitcoin or Freicoin? Freicoin obviously doesn't have segwit until we complete rebasing and activation. Segwit is fairly well described in the bitcoin developer documentation on bitcoin.org, but probably the best thing to do is to switch over to an actively maintained bitcoin block explorer. I don't have a specific recommendation but I know here are several. 

With a 1wp we cannot expect all coins to move to the new chain -- some wallets are lost for example, so when do you stop supporting old UTXOs? With a spinoff or hard fork it would be possible to inherit the old UTXO set, but at the cost of not being able to deprecate handling the old chain data structures. A sync from genesis requires understanding the old chain up to a certain height, then switching to code that handles the new chain. Alternatively you can dump the UTXO set and import it at the changeover height, but do you still handle old style transactions at that point? Do you flag day expire them so at some point you just drop compatibility? What date do you choose for that?
None of the choices there seem obvious, clean, or desirable. The 1wp with dual subsidies is a middle ground. You allow people to voluntarily move coins over and the new chain code never has to understand the old one except as is required for the peg code. You limit the maximum transfer however to something slightly larger than the current issuance. It will take some time for subsidy to play out on the old chain enough to exhaust that peg in limit -- longer if there are substantial lost coins or if people are asleep at the wheel. During that time both chains are operational. But once it is done you can remove the pegin and legacy transaction handling code from the new chain, and the old chain dies off (because subsidy can no longer be transferred to the new chain so it loses value).
As mentioned by Skaro it is relevant to this thread because it achieves something close to option 3, but in a regulatory compatible way while also incentivizing dual use of the chains during the upgrade process, and solving the old UTXO problem once and for all.