Last year I spent a month in San Francisco's sunny Silicon Valley on a business trip. I was in charge of bringing up a number of boards straight off the manufacturing line. Unfortunately there were a few problems with the boards, which left us baffled for a few days until bubbling was mentioned. This is when I found out all about MSL (moisture sensitivity level).
On specific ICs, mostly CPLD's, FPGA's and other specialised semiconductors, the parts need to be baked at 125°C for a minimum number of hours based on the reflow temperature specific to the PCB. Now I don't know much about the whole process, but from this trip, I've learnt a little bit more. If these parts aren't baked for the specific time, bubbling, cracking, blistering, internal shorts and other related errors can occur due to the expansion of the moisture during reflow, which is called de-lamination. Most of the time, these problems may not even be visible and could take weeks of testing to find the part doesn't behave as it should. In the extreme cases, the bubbling, cracking and blistering mentioned above occur, which is commonly known as the "popcorn" effect.
There are a number of moisture sensitivity levels described on Wikipedia's page. The sensitivity level determines the time allowed for a device to remain out of its packaging before it needs to be baked again to remove moisture. Resistors and capacitors usually have a moisture sensitivity level of 1, which is infinite, where as most FPGAs or CPLDs have a level of 6, which means a mandatory bake before reflow. The one thing to note here when receiving parts from a distributor is that the parts should arrive in a vacuum sealed anti-static bag with dessicant and a moisture indicator card. These parts tend to be more expensive than the usual parts.
Trying to debug issues caused by delamination is near impossible and often it is just best practice to replace all questionable parts.
This one shortcut compromised the whole batch of boards and cost thousand's of dollars and weeks for our customer, but replacing each and every one of them would be even more costly and timely. From what I've seen, this is the quickest way to lose a manufacturing contract and another example of trust the manufacturer, but verify!
