arithmetic-pitfall

Avoiding arithmetic pitfalls

Do division last

Division has that unique power to turn two perfect integers into floating point numbers. If possible, put divisions at the end of an equation. Here's why:

(1 / 2) * 10
0

Of course, this is entirely dependent on your languages ability to deal with type conversions on the fly. The issue above is that 1 / 2 = .5, which because the division was done between two integers, an integer result of 0 is returned. This may or may not be an issue in your language. But, had you done it this way:

(1 * 10) / 2

You would get the desired result, no matter what language you're using. Of course, this is largely dependent on the problem at hand, and this approach won't always steer you clear of errors. But, it's a good general best practice when using equations.
 

Use high-capacity data types for operations on unvalidated user input

That was a mouthful, so let's look at an example:

You created a financial calculator which, among other things, allows a user to multiply two numbers together. Your calculator only has room for 9 figures, so assume that an integer data type is more than enough to store calculation results. The user then tries to multiply: 1,000,000 x 1,000,000. Depending on your language, this far exceeds the integer data type storage space. What happens also depends on your language. For some, you get an error. For others, you'll get an entirely inaccurate result, which does fit into an integer data type.

In the case of a financial calculator, a user would much rather their program crash because your large data type caused an out-of-memory error, than return an inaccurate result.

 

Get a handle on floating point operations

function safeAdd(op1, op2) {
  var maxSigFig = 0;

  if (op1.toString().indexOf('.') !== -1) {
    var sigFig1Length = op1.toString().split('.')[1].length;
    if(sigFig1Length > maxSigFig) {
        maxSigFig = sigFig1Length;
    }
  }
  if (op2.toString().indexOf('.') !== -1) {
    var sigFig2Length = op2.toString().split('.')[1].length;
    if(sigFig2Length > sigFig1Length) {
        maxSigFig = sigFig2Length;
    }
  }
  
  return maxSigFig;
}

safeAdd(1,2.34);

Life was so simple a million years ago. That is, before a clumsy caveman stepped on a stick, broke it, and discovered fractions. Humans typically don't struggle with the concept of fractions, but to computer processors, they just don't make sense. Of course, layers of abstraction were added on top of the CPU to assist with operations on floating point numbers, but, to this day, we still get unexpected results.

I've never come across an application where, when adding two floating point numbers, a result with more precision than either of the two operands is necessary. Yet, it happens by surprise regularly. My favorite example is the online shopping cart. If you're lucky enough to have ever created one "from scratch", you've probably encountered a shopping cart like this:

Item 1    $29.95
Item 2    $14.95
Total:      $44.905

Imagine the response of your customer – "are they really trying to cheat me out of 1/2 of a cent? Sounds kinda like the scam they pulled in the movie Office Space." But then again, the 1/2 cent was in regular circulation until the 1850s (I'm not lying, check it out! http://en.wikipedia.org/wiki/Half_cent).

So, who do we blame here? The processor? Not really, it's not the processors job to make a judgement call about your application. Blame the language compiler or interpreter?  Maybe, but does that do us any good?

My thinking is to introduce a function to your application, one that does make assumptions about your decimal precision. So, here's a little JavaScript function I came up with that will take two operands, add them together, and ensure that the result is not any more precise than either of the two operands.

function safeAdd(op1, op2) {
  var maxSigFig = 0;

  if (op1.toString().indexOf('.') !== -1) {
    var sigFig1Length = op1.toString().split('.')[1].length;
    if(sigFig1Length > maxSigFig) {
        maxSigFig = sigFig1Length;
    }
  }
  if (op2.toString().indexOf('.') !== -1) {
    var sigFig2Length = op2.toString().split('.')[1].length;
    if(sigFig2Length > maxSigFig) {
        maxSigFig = sigFig2Length;
    }
  }
 
  var total = op1 + op2;
 
  return parseFloat(total.toFixed(maxSigFig));
}

safeAdd(1.31,2.49);
3.8

The function above is great, but it makes the assumption that your language will have a built-in function for rounding to certain number of decimal places. In the case of JavaScript, that function is toFixed(). toFixed() also has the unfortunate side-effect of converting your number to a string, forcing us to cast it back to a floating-point value. The function below doesn't make any such assumptions, and does not require any casting from string to float.

function safeAdd(op1, op2) {
  var maxSigFig = 0;

  if (op1.toString().indexOf('.') !== -1) {
    var sigFig1Length = op1.toString().split('.')[1].length;
    if(sigFig1Length > maxSigFig) {
        maxSigFig = sigFig1Length;
    }
  }
  if (op2.toString().indexOf('.') !== -1) {
    var sigFig2Length = op2.toString().split('.')[1].length;
    if(sigFig2Length > maxSigFig) {
        maxSigFig = sigFig2Length;
    }
  }
  
  var total = op1 + op2;
  
  var mult = Math.pow(10, (maxSigFig + 1) - Math.floor(Math.log(total) / Math.LN10) - 1);
  return Math.round(total * mult) / mult;
}

function safeAdd(op1, op2) {
  var maxSigFig = 0;

  if (op1.toString().indexOf('.') !== -1) {
    var sigFig1Length = op1.toString().split('.')[1].length;
    if(sigFig1Length > maxSigFig) {
        maxSigFig = sigFig1Length;
    }
  }
  if (op2.toString().indexOf('.') !== -1) {
    var sigFig2Length = op2.toString().split('.')[1].length;
    if(sigFig2Length > maxSigFig) {
        maxSigFig = sigFig2Length;
    }
  }
 
  var total = op1 + op2;
 
  var mult = Math.pow(10, (maxSigFig + 1) - Math.floor(Math.log(total) / Math.LN10) - 1);
  return Math.round(total * mult) / mult;
}

It's not likely that this type of function would be used in client-side code, but I chose JavaScript for my example because it's well known and understood. Contact me if you'd like me to write the same function in a server-side language, it'll only cost you 1/2 cent :)

Why follow me on Twitter?

  • I tweet about new technologies, services or libraries I find interesting
  • Yeah, sometimes I'll post a pet-peeve or rant about something trivial
  • If I discover something that made my web development life easier, I share it
  • I'll shout out any handy tip that I think might be useful to other devs


Tagged , , .

Updated: 2012-11-15

Phil LaNasa follow us in feedly