Few thoughts, blunders, lessons on IT

Some of my favorite blunders (we learn from our mistakes):

• On my first day on the job that I was all by myself as a Laboratory Teaching Assistant, I crashed the main VMS server. No one could log in. I worked for hours off the clock to get it going. I thought I was optimizing it.

• I have bricked an Ascend Terminal Server. I had to go on-site thirty miles away and Ascend support got it going through a 9600 baud modem to upload the thin firmware first. Then, I was able to upload the fat firmware to get it going. Thin load only works on the local network.

• I stopped work at an S&P500 company's design centers. An Engineer asked me not to send a section of Open-Source directory Branch, and I was tired and was at the end of the shift. I made an error in the rsync config. I ended up deleting all directories overnight for the company's open-source libraries. It took three to four days to reupload the libraries to the design centers over the corporate wan.

• I used to compile my open-source code. Most distributions are way out of date, too many, or not suitable options. I took down a mail server because of cascading issues with MySQL and OpenSSL updates (change in .so files).

Some things that tweaked my mind.

• Many times my job was to keep old equipment alive. Computers that were ten years old were used to create, test technology that did not come to market for another five to ten years into the future.

• COBOL, Fortran is still very much alive.

• Batch processing drives most real-time computing: If you have a background task that is completed very fast, most people cannot tell.

Over the years, I learned a few lessons.

• Document both in code and in a document system. You will have to remember what you did over six months ago.

• Do not get enamored with the hottest computer language or software. Use the tool that gets the job done and what the company uses. Use tools that leverage your workmate's knowledge. Next time you go on a vacation or in a middle of an important project, your workmates can help you.

• Revision control and backups is your friend.

• Tell some else what you are doing. Don't work in the dark.

• Sometimes, there are just no points for style. If you take a month to accomplish something, no matter how cool, if the job was needed in a week, for the most part, who cares. You did not get your job done.

• If you make a mistake, confess your sins immediately. Then explain how you are going to get it fixed.

• There has to be a balance between performance and stability. A system that is too slow is unusable. At the same point, a system that is very face and not stable is worse.

• The best computer language is what is used at your workplace. It is fine to learn new stuff at home, but you cannot be the odd man out.

• Uptime is important.

• Monitor. You want to know when things go wrong before your users report problems.

The above list is not all-inclusive, and some lessons I learned the hard way.