Author Archives: Thomas Gouverneur

How to upgrade a Solaris server to a particular patching level

Situation Today at work, I needed to prepare the patching of a two-nodes cluster. Not only I should patch this cluster, but I should also mimic the same patching level as the currently used prod server. Well Well. Instead of … Continue reading

Posted in WeSunSolve | Leave a comment

sfpC.pl – The Solaris Fingerprint Companion revived!

A few ago, I’ve found the sfpC.pl script from Glenn Brunette which was used to check the fingerprint of any Solaris file against the SunSolve Fingerprint Database. I’ve managed to port this script to be used against WeSunSolve fingerprint database, … Continue reading

Posted in WeSunSolve | Leave a comment

My Oracle Support: Authenticate with CLI

I was wondering since some times how I could authenticate on MOS using CLI. Now I got my answer: By reversing the whole SSO auth process in a python script that generate a cookies.txt file, usable with wget. I know … Continue reading

Posted in Solaris | Leave a comment

WeSunSolve.net: 6 Months later

6 Months ago, I created the “sunsolve.espix.org” website running at home on my personal VDSL link; 6 Months later, the website is still evolving. Where we started Everything started when SunSolve.sun.com has died. I remember my frustration at that time.. … Continue reading

Posted in WeSunSolve | Leave a comment

Solaris 10u10 is now out !

And there it is… Solaris 10 update 10 ISO are now available for download. Get them here Happy live upgrade !

Posted in Solaris | Leave a comment

Solaris 11 Early Adopters release.

Yesterday, Oracle announced the Solaris 11 Early Adopters release, a new bunch of ISOs are available for download, only for Gold members of Oracle/SUN. Anyway, if you’re a lucky one, you can have it here. Rumors said in the wild … Continue reading

Posted in Solaris | Leave a comment

ZFS Fragmentation issue – examining the ZIL

During the past days, I’ve been troubleshooting some Zpools with heavy databases usage. The issue reported by the customer was a huge performance decrease after the last reboot of the cluster. The issue has been finally troubleshooted and has been identified as being a ZFS Fragmentation problem. I’ll try now to share the lesson learned with this issue.

First of all, the performance problem has been investigated with a drill down method, we used teamquest to visualize the differences between now and the week before, what we discovered is that the I/O on the pool which holds the DBF of the database had literally exploded. We had sometimes more that 50K write iops balanced accross the different vdev of the affected pool.

The pool was actually configured like this one:

 # zpool status i-ora-pro06-dat1-pl pool: i-ora-pro06-dat1-pl state: ONLINE scrub: none requested config: NAME                                       STATE     READ WRITE CKSUM i-ora-pro06-dat1-pl                        ONLINE       0     0     0 mirror-0                                 ONLINE       0     0     0 c6t60060E800571FC00000071FC000020C3d0  ONLINE       0     0     0 c6t60060E800570FB00000070FB000020C3d0  ONLINE       0     0     0 mirror-1                                 ONLINE       0     0     0 c6t60060E800571FC00000071FC000020C4d0  ONLINE       0     0     0 c6t60060E800570FB00000070FB000020C4d0  ONLINE       0     0     0 mirror-2                                 ONLINE       0     0     0 c6t60060E800571FC00000071FC000020C5d0  ONLINE       0     0     0 c6t60060E800570FB00000070FB000020C5d0  ONLINE       0     0     0 mirror-3                                 ONLINE       0     0     0 c6t60060E800571FC00000071FC000020C6d0  ONLINE       0     0     0 c6t60060E800570FB00000070FB000020C6d0  ONLINE       0     0     0 errors: No known data errors

The SAN disks behind have been able to handle a lot of I/O and the SAN was also checked for any problem, but, clearly, the problem was the heavy IOPs load on the LUNs.

Following to this, we’ve ran a zpool iostat -v i-ora-pro06-dat1-pl 2 for a while, to confirm what we were thinking. This confirmed the heavy write load on the vdev.

Still with teamquest, we were able to see that the kind of write operations that were done on disks, were actually very tiny write blocks.

We have then opened a support case at the Oracle-SUN’s support and uploaded some Guds traces which exposed the problem we faced. Here are the complete explanation of this problem as well as the way to detect it and also the fix.

Continue reading

Posted in Solaris | Leave a comment

sunsolve.espix.org : A new tool for Solaris sysadmins…

When the merge of Oracle and SUN became reality, we lost one of the greatest documentation portal for Solaris: Sun Solve. While I’m a Solaris sysadmin myself, I needed a tool to manage my daily patching, to ease search with … Continue reading

Posted in WeSunSolve | Leave a comment

PHP5: Memory leak with object arrays

As I’m coding with PHP since years now, I’m used to the language. I’m doing now some website in PHP with part of their processing into CLI php scripts, which are sometimes manipulating high amount of data.. This was the … Continue reading

Posted in Programming | Leave a comment

Solaris 11: Dual IP Stack zone (IPv4/ IPv6)

I’ve recently migrated my workstation under OpenIndiana, which is based on the latest release of OpenSolaris, and thus, on Solaris 11 express. I wanted to migrate some of the services I was hosting into zones, but as theses services were … Continue reading

Posted in Solaris | Leave a comment