Credit Cards Failing Open
Most consumers are aware that when you close a credit card account, it’s not really closed. For “convenience” reasons, recurring subscription charges such as your cable bill will continue to be approved. You can kind of see where the credit card companies are coming from, but it’s a pretty weak argument. The cable company just needs to notify me that the credit card on file is no longer valid, and I’ll update my information. Problem solved.
But that credit card weirdness is nothing compared to the one I’m about to describe.
Before we do that, let’s take a moment to discuss the design principle of failing securely. The general idea is that if a security mechanism fails, it should fail closed. If your firewall crashes, it should block all traffic, not allow all the packets through. If the power source to your card key system is interrupted, it shouldn’t unlock all the doors. If the connection between your application server and your LDAP directory is severed, subsequent authentication requests should be rejected, not approved. This is not rocket science.
So back to credit cards. I had a conversation last night with an old friend who related a bizarre situation they had encountered during the QA process for one of their web applications. One of their tests involved repeatedly attempting a credit card transaction using a canceled/expired American Express card. Here’s what they saw in their logs, paraphrased by me:
Attempt 1: Denied Attempt 2: Denied Attempt 3: Denied . . . Attempt 49: Denied Attempt 50: Denied Attempt 51: Approved
What the…? Approved? That can’t be right. So they ran the test again. Every time, after multiple consecutive rejected attempts, the transaction would inexplicably go through. The threshold wasn’t always 50, but the general pattern was consistent — keep trying and eventually it’ll work. Clearly, this had to be a bug in the code, but a deep-dive into the guts of the application turned up nothing. The application security group got American Express on the phone to see if they had any insight on this odd behavior. The answer? They didn’t concede the failure was on their end, despite log data showing the successful authorization codes.
My gut instinct would be that the application requesting the transactions wasn’t failing securely (e.g. network connection to AmEx timed out, so just approve the transaction). But that explanation wouldn’t account for authorization codes coming back.
So what in the world is going on here? Why would the system behave this way? Is it by design? I can’t think of a single legitimate use case for failing open like this. If this is actually a design decision by the credit card companies, I have no doubt that someone in our audience knows the rest of the story.