A Sandbox/Test Server Is a Sandbox For A Reason

That's right. Read it again.

"A Sandbox Server Is a Sandbox For A Reason." Now, read it again, and again, and again. MEMORIZE IT.

What is a Test/Sandbox Environment?

A test/sandbox environment is a testing playground for the developer/user (in the context of this conversation, its developer). Its an exact copy of the production environment, but does not affect anything in production. It allows the developer to test code before it goes into production. This allows the developer to find errors before they actually cost the company any money. Test/Sandbox environments are an integral part of software development and service providers. They make sure that the quality of the product released is top notch (of course this is assuming proper testing is taking place).

This is nothing new, its not rocket science. Everyone tests before they ship a product (well, anyone with a brain does at least).

When you're working with a service provider you expect them to have a test/sandbox environment that matches there production environment. Tons of service providers have these test servers, here's a few:

• Google AdWords Preview Tool - Lets you preview searches using Google so you can preview your ad before it goes live.
  • Benefit - You don't get penalized for impressions or clicks. No money is spend, no money is transferred.
• Google Checkout Test Server - Allows you to post test transactions.
  • Benefits: You can test without being charged or have actual money transferred.
• PayPal SandBox - Lets you test your Pay Pal transactions.
  • Benefits - You can test without being charged or have actual money transferred.
• Amazon Mechanical Turk SandBox - Allows the users to test their applications and HITS.
  • Benefits - You don't loose money nor put up a HIT that could be worked on. (You pay for HITS). No money is transferred.
• Amazon Flexible Payments Service (FPS) - Allows you to test the payment systems you set up.
  • Benefit - Money is not actually transferred.
• SalesForce Sandbox - Allows developers to post test transactions to the Salesforce Service.
  • Benefits - Eliminates costly development mistakes. Helps save on testing, therefore eliminating money being transferred for a development task.
• Virtual Machines (Virtual PC, VMWare) - The big daddy of them all. Test your entire environment before you put it into production.
  • Benefits - Eliminate HUGE costs associated with lack of testing/deployment scenarios.

These sandbox environments share a common goal - THE GOAL OF ELIMINATING COSTLY MISTAKES

How? Simple. If developers are given a test environment, they can test their code without worrying about messing up production environments. If these services involve money, its VERY IMPORTANT that the said service has a test environment.

A real life story...

Recently I was working on a side project that involved a service provider (none of the listed above) and I found their test server to be quite useful. I could post transactions to it, and do all the testing I needed to. A few weeks later the production site changed, yet the test server remained the same. Not only is this irresponsible as a service provider, its a risk that the service provider puts on themselves. They have now put themselves in a position where the service might now work, and some users might abandon the product. Upon asking the service provider when the test environment would be updated to mimic the production environment I got this response (edited to make sense in this context):

"There's really no point in having an exact copy of production since it's only really designed for practicing entering form data. Our live environment changes daily (along with other criteria) and, other than changes to form validation or form fields, these changes are never reflected in the test environment (which is also why there are some different types in the test environment that don't exist in the live environment)."

Huh? What!?! Lets go over a snippet of this... the first part really makes my jaw drop.

"There's really no point in having an exact copy of production since it's only really designed for practicing entering form data."

Whoa, DANGER, WRONG WAY, TURN BACK, for real... for real. Ok, well, the last time I checked when you have a test environment - its a test/sandbox environment for a reason: It mimics the production environment to allow the developer to perform tests without affecting anything in production. So why is there no point? If its not the exact same thing, then how am I to be sure its going to work the same. That's right, I WONT KNOW.

Unfortunately this service provider has not updated their test server to this day. It's a service that I need to use, but I will openly admit, I think its a bad business decision on their end. I've had to perform all kinds of trickery to validate my code does what is supposed to do, but still, I'm only 65% sure its right. Which sucks, no test server, no way to test everything.

This is a lesson that all service providers and developers should note...

A broken test environment is like having no test environment at all.

Just imagine, if you can't compile your code, you can't ship your code. So, if you cant test your code properly, how can you be sure its doing the right thing? You cant.

Service providers: Make sure you  have a valid test environment that mimics production, EXACTLY.

Developers: Demand a test environment. Ruthlessly test your code. Unit test to hell. Functional test your code like hell.