PHP5: Memory leak with object arrays

As I’m coding with PHP since years now, I’m used to the language. I’m doing now some website in PHP with part of their processing into CLI php scripts, which are sometimes manipulating high amount of data.. This was the case with one of my scripts, which was loading 28.000 object from a database, to treat them one by one.

After the load of the object, I was using a foreach to loop through the array and at the end of each iteration, I was actually unsetting the current object and removing it from the array, to allow memory to be freed after each object was treated. Unfortunately, it didn’t worked. I was running out of memory very quickly.

I’ve then found the issue, by using a little test script like this:

 <?php class Test { public $id = 0; public $var1 = ""; public function __construct($id) { $this->id = $id; } } $myarray = array(); for($i=0;$i<25000;$i++) { $v = new Test($i); $myarray = $v; } $mu = memory_get_usage(); echo "- Memory usage: ".number_format($mu) . "\n"; foreach($myarray as $k => $v) { $v->var1 = "mystring1"; unset($v); unset($myarray$k); $m = memory_get_usage(); $diff = $mu - $m; echo "- Difference: ".number_format($diff)."\n"; } ?>

As you could see if you run this little script in CLI, php memory usage’s growing after each iteration, despite the unsets. The problem comes from the “foreach”, which seems to cache the array before iterating, this hold references to the object that aren’t getting freed. To fix the issue, replace the loop with the following code;

 for($i=0,$c=count($myarray);$i<$c;$i++) { $v = $myarray$i; $v->var1 = "mystring1"; unset($v); unset($myarray$i); $m = memory_get_usage(); $diff = $mu - $m; echo "- Difference: ".number_format($diff)."\n"; }

And there it is, now php free correctly the objects after each iteration.

Posted in Programming | Leave a comment

Solaris 11: Dual IP Stack zone (IPv4/ IPv6)

I’ve recently migrated my workstation under OpenIndiana, which is based on the latest release of OpenSolaris, and thus, on Solaris 11 express. I wanted to migrate some of the services I was hosting into zones, but as theses services were using dual stack IP addressing, I needed to assign both v6 and v4 ip addresses to the zone.

Here is how it could be achieved:

# zonecfg -z vz01 # info net: address: 2a01:38:8007:aaae:a::1/64 allowed-address not specified physical: vlan200 defrouter not specified net: address: fe80:aaaa::0001/10 allowed-address not specified physical: vlan200 defrouter not specified net: address: 10.42.2.151 allowed-address not specified physical: vlan200 defrouter: 10.42.2.254 

So, to give a little explanation on this: We have first assigned an ipv4 address into our vlan200. This is old stuff. Then, we assign a local-link ipv6 address to allow the zone to reach the default ipv6 gateway of the global zone (Which is a link local adress). Then we just assign the second IPv6 public address, without specifying a default router as the global zone is inside the same ipv6 range and has already a default gateway for this ipv6 range. Then, just boot the zone and try!

This work like a charm:

root@vz01:~# ping -A inet6 ftp.belnet.be ftp.belnet.be is alive root@vz01:~# 
Posted in Solaris | Leave a comment

Oracle 11gR2 with Sun Cluster 3.2

As stated inside the Sun Cluster 3.3 releases notes, this version adds support for Oracle 11g Release 2. Although, there is some case where upgrade to 3.3 is harder than only patching existing 3.2 installation.

Here it is!, looking at the changelog of the following patches, they add support for 11gR2 to the Sun Cluster 3.2 release:

Sparc

 - 144221-01 - 126047-15

X86

 - 144222-01 - 126048-15

Oracle’s new release could then be clusterized without upgrading to 3.3. Notes that it is although recommended to go on with Solaris 10 Update 9 for new Oracle releases.

Posted in Solaris | Leave a comment

Live Upgrade: Still on track!

Just updated a cluster of Solaris 10u8 with SUN Cluster 3.2 to U9 ! It worked like a charm!

Next step… upgrade everything to SUN Cluster 3.3…

Posted in Solaris | Leave a comment

BBox 2: For fun and profit

From time to time, I used to play a little bit with my modem-router which is provided by my ISP (Belgacom @ Belgium). The bbox2 or Sagem F@st 3464 is after all, a great piece of hardware locked up by belgacom.

After some time, I managed to connect to the telnet and gain root access. Fine, let’s upload some binaries like a real busybox and some other things (incl: tcpdump, iwconfig, …). Now I have a box I can play with a bit.

Sad story: almost every 2 days, belgacom is updating the box and you end up with all your files being send to /dev/null.

I’ve then decided to write some script to put back my files in place easily when such updates are done.. OK, fine, but what would happen if belgacom decide to fix remotely the vulnerability that allow us to gain access to the box ?

We would then being forced to find another vuln, loosing a lot of time.

So, we could all learn that Belgacom is managing its bbox with the TR-069 protocol… What if we disable this daemon ? Cool. let’s go: I’ve stripped down the process of the box to the minimum: exit tr69, exit tr98, exit sipd! Good, now the box is running without updates, but even better: I have now more than 80% of the CPU and memory freed!

As I’ve spent some time writting theses scripts and compile theses binaries, I wanted to share them with you, together with an usage example. So you could find theses here

Hope it helps!

Posted in Programming | 1 Comment

m0n0wall-CMI: The future of a great project…

Sometimes ago… I’ve started the m0n0wall-cmi project… It was apparently something that a lot of people were missing and it found some interest in the community.. Unfortunately, I had no time to continue the project further and so it kinda “died”…

Now, I’m wondering if there is still some interest in this.. and maybe I’m considering the fact to get back to work 😉 The goal would be also to adapt the whole thing to be more “web 2.0” (for whatever that means :)) and compatible with all the m0n0wall-like appliance (yes, indeed, pfsense…).

So, if you’re still interested into this project, please, shoot a comment here and let your contact details 😉 If I anyway start again to work on it sooner or later, I would be glad to have some alpha-testers under the hand..

Cheers!

Posted in Programming | 1 Comment

Solaris SUN Cluster: Live update

As I’ve recently tried the patching of a sun cluster with Zones/ZFS/global filesystem and everything running, I’ve noticed that not so much people have tried this, so I might share the experience here 😉

So read the complete article to see how it behaves 😉

Continue reading

Posted in Solaris | Leave a comment

Solaris Sun cluster core patch 126106-40 issue

I’ve recently had to apply the 126106-40 core patch for SUN Cluster and had some issues with it.. Seeing on the Internet, some issues with this patchs are documented by Juergen Scheich’s blog and describing a howto fix some of them. Actually, the error I had was a bit different from the “loaddid” cycling dependencies that was found on his blog. I had a similar issues with the sysidtool:net service which gave me the almost same error as the loaddid did. No pun intented 😉

Applying the same procedure as found on Juergen’s blog fixed also that issue! So you can just:

  1. Boot in non clustered mode
  2. Remove the loaddid service from SMF: svccfg delete loaddid
  3. Import it again from the patch’s XML file: svccfg import /path/to/patch/loaddid.xml
  4. Refresh the service: svcadm refresh loaddid
  5. init 6

This has not been faced on all the node of the cluster I was patching, but I thought it was worth a note here 😉

Posted in Solaris | Leave a comment

Mobotix Cameras: getting the stream

I’ve grabbed some times ago two Mobotix cameras.. I must say first that I was impressed by they quality (nice look, high resolutions, good framerate and nevertheless, GNU/Linux based).

As I put them to monitor some outdoor part of my house, I’ve wanted to record everything that would happen, but hey, first question, how to record ?

Indeed, they don’t support MPEG4 streaming. In fact, they are using some half-proprietary MxPEG format, that seems to be on its way to become a good choice when it comes to camera streaming. I’ve searched out the internet to find the SDK of the camera that provide also the little mxgconv binary, allowing you to convert MxPEG to JPEG frames. Well.

Here is then the little script I used to record in MPEG4 everything coming from the cameras on hourly-basis.

Continue reading

Posted in Uncategorized | 2 Comments

Inject Outlook PST Archives files into IMAP account

After searching for a LONG while for a method to inject my outlook pst files into my IMAP server without any success… I decided to write some scripts to do it myself. This is using a mix of libpst, perl, bash and some mbox2imap script found on the net…

Check the full article for details!

Continue reading

Posted in Uncategorized | Leave a comment