Charl van Niekerk » Blog: October 2007

Main

Latest

Gmail now supports IMAP!!!

Yes, the title says it all. I have been waiting for this for a really long time now (basically since they launched POP3 access)! I want to keep all my mail (including my sent mail) on the server so that I can access it from multiple locations but through my mail client while I'm at home. I also don't want to start downloading each attachment that gets sent to me automatically (being in South Africa on a bitcap).

So you just need to enable it in your Gmail Settings and then configure your mail client accordingly. Check out IMAP Access in the Gmail Help Center for more info.

MD5 and SHA-1 Password Hashing in PHP / MySQL

It is pretty standard at the moment for PHP-based CMSes to use MD5 hashing for user login passwords that are stored in a database. This means that even if somebody gets read-only access to the database, they still can't log into the site. (The point behind hashing is that it's one-way only - hashes should not be able to be decrypted.) The reason for MD5's popularity is partially because PHP just makes it so easy. However, I only discovered today that it's just as easy to do SHA-1, which seems to be generally considered as substantially stronger. Take a look at the PHP sha1 function documentation.

I just wrote this little shell script to generate SHA-1 hashes on the go (it's just copied from another script I wrote earlier for generating MD5 hashes):

#!/usr/bin/php
<?php
if (count($_SERVER['argv']) < 2) {
  echo "Error: At least enter something to hash, noob!\n";
  exit(1);
}
foreach ($_SERVER['argv'] as $i => $text) {
  if ($i) {
    echo "$text\t" . sha1($text) . "\n";
  }
}

If you would like to learn more about MD5 and its problems, check out MD5 Vulnerabilities on the Wikipedia.

So, are there any reasons not to migrate? Time to change the user/login management classes and change the CHAR(32) database field to a CHAR(40), because the "normalised" strings the sha1 PHP function returns by default are 40 characters in length instead of the usual 32 for MD5 hashes.

Migration of existing databases will be a big problem though, as you can't (or shouldn't be able to) decrypt the existing passwords for re-hashing. You could probably hash the MD5 hashes with SHA-1 (thereby double-hashing) though if you really like, or try your luck with large MD5 rainbow tables. Maybe you get lucky and all your users are noobs with < 8 character passwords (just kidding). Please note that SHA-1 (such as any other hashing function) is also still vulnerable to rainbow tables.

One last point I wanted to address, although this is somewhat off-topic. Why do people use VARCHAR field types in MySQL databases for storing hashes if they will always be the same length? Isn't it more efficient to just use CHAR? I believe the latter is more efficient in cases like this - please see the comments on the page The CHAR and VARCHAR Types for more information.

Update: Peter Rotich pointed out to me that a simple strategy would be to allow both MD5 and SHA-1 hashes in the database. A user's MD5 hash is kept and as soon as he logs in using the correct password matching up with the MD5 hash, the password is hashed again using SHA-1, stored and the MD5 hash is deleted out of the database. Next time the user logs on, their password is checked against the SHA-1 hash. Smart one, Peter, thanks!

Opposites

Earlier today I got an link through Muti to a page of funny exam answers. My favourite must be this:

This must have been an uber-leet gamer boy/chick!!! :D

Cape Town - Here we come!

Ok I don't really want to make yet another post but basically this is it! I'm leaving with Jayx tomorrow for Cape Town. We'll be meeting lots of interesting and exciting people so I am really looking forward. I am not going to be online for the few days (well, I doubt it strongly... but who knows... computers seem to follow where I go... must be the circles I move in).

If anybody wants to try and get a hold of me urgently please phone me at 072 405 8378. If you send email I will probably handle it Monday afternoon or later.

This will be my first 27 Dinner so I'm even more excited.

Coolness people! Have a good one! Hope you don't miss me pesting you online too much... ;)

Also, if you sent me e-mail and I didn't reply yet, I unfortunately got a little behind the last few weeks due to the exam pressure. Please resend if it's urgent but otherwise I'll try to get around to it as soon as possible.

Microsoft DNS Server

This is actually really ooooold shit I am posting now but hey I figure now is as good a time as any to get these out.

Anybody that knows me will also know that I hate Windows servers. I like efficiency, power, flexibility and the ability to automate virtually anything.

Microsoft DNS Server is just one of those things that really frustrates me. They seem to expect you to have to do everything using their GUI. They don't provide you with an API but at least they provide you with a separate optional add-on command line interface as part of the Windows 2000 Service Pack 4 Support Tools and Windows XP SP2 Support Tools for Advanced Users (nice they think I'm an "advanced" user - probably all *nix people are relative to Windows norms) called dnscmd. For example:

dnscmd /ZoneAdd mydomain.example.org /Primary /load

This is not so bad as now one can write some scripts to make use of this. This is obviously still not nearly as clean as a proper API though.

Anyway, on the machine that I was working on, the DNS database files were located at %systemroot%\system32\dns.

Each domain seems to have its own file. This is good.

My dilemma was this. I had a list of about 20 domains I needed to import. Each domain's records would be identical (at least, for a start). There were quite a few A, CNAME and MX records.

Firstly, I create myself a "template" file. All I did was to simply set up the first domain. I then went to the above-mentioned directory and got the right file. This I will use as a "base".

If I would do this again, I would naturally write Python scripts. However, at the time I was still a bit of a Python noob, so I decided to write some PHP shell scripts, which is also fair enough I guess as this is pretty simply stuff.

I simply create a text file with all my new domains, each domain on a new line, for example:

mydomain1.example.org
mydomain2.example.org
mydomain3.example.org

Of course my file was like much larger.

Anyway, so then I wrote a small PHP script (actually, I don't even think this qualifies as a "script", just as a quick-and-dirty hack).

<?php

$filename = "template.txt";
$handle = fopen($filename, "r");
$template = fread($handle, filesize($filename));
fclose($handle);

$domains = file("domains.txt");
$cleaned = fopen("domains.txt", "w");

foreach ($domains as $domain) {
    $domain = strtolower(trim($domain));
    if ($domain) {
        fwrite($cleaned, "$domain\n");
        $zonefile = str_replace("mytemplate.example.org", $domain, $template);
        $handle = fopen("output/$domain.dns", "w");
        fwrite($handle, $zonefile);
        fclose($handle);
        echo "$domain zonefile created\n";
    }
}

fclose($cleaned);

As you can see my template file is called template.txt and my list of domains is saved in domains.txt, all in the current working directory. Here the mytemplate.example.org would be the first domain you set up for the template. You'll probably want to change that. Note this is for the file contents, has nothing to do with the file name!

Also note the cleaning the above script does on domains.txt. This is because I copied and pasted the list of domains from an e-mail and it turned out a bit messy.

So just create a directory in your current working directory called output (or whatever) and execute this as a shell script. The easiest is probably php import.php (or whatever you called the file). Otherwise you can do the proper shell script thing (with the #! and chmod +x).

Anyway, now you have all your new zone files created. I did all of this on my Linux box so just transferred all those files in the output directory back to the Windows server.

However, even though the right files are now in %systemroot%\system32\dns, you still need to get Microsoft DNS Server to recognise them. This is not as simple as it should. Being a *nix geek, I immediately restarted the DNS Server background process through using Windows "services". Didn't help... I now need to still add each domain using the new zone files manually.

At that time I didn't know about the command line tool so I did this using the GUI. Really frustrating. Otherwise I would have had another script to show off now. I would have had to run this on the Windows box (never tried to get the PHP command line interface to work on Windows yet). Otherwise I would have been able to use the example dnscmd command earlier in the post to add the individual domains easily and thereby fully automating (well, at least almost-fully-automating) the process.

I'm still trying to figure out what the purpose of this post is, but I guess at the end of the day, I just want to vent my frustration with applications that provide a GUI and no API or command line tools. They just make the life of a coder so much easier. I can't stand repetitive tasks.

The obvious question you have is probably "Did you really save time?" Maybe a little, maybe not, but at the end of the day at least everything worked first touch because the amount of room for human error has been reduced and I had a lot more fun than otherwise. And I also got to do some fun research.

RFC 3339 Timezones in Joomla! 1.5

I wrote the following piece of code for my GSoC project this year but ended up not having to use it. Just found a backup copy of it on my hard drive now and thought I should post in case it might be of any use to anybody.

This generates the current timezone for use in RFC 3339 dates, obviously for use inside of Joomla! 1.5 but since it's commented I'm sure it can be easily adjusted for your needs.

This is a bit of a manual calculation but you can probably get what you need from the Joomla! 1.5 API.

Please note that this has never been tested properly so use at your own risk. :)

<?php

// Get timezone from configuration
$config =& JFactory::getConfig();
$offset = $config->getValue('config.offset');

// Split up the offset into hours and minutes according to the decimal point
list($hours, $minutes) = explode('.', $offset);

// Calculate the minutes according to the fraction of 60 minutes
$minutes = 60 * "0.$minutes";

// If the hours is non-negative, add the positive sign
if ($hours >= 0) {
  $hours = "+$hours";
}

// If the hours is represented as a single numeric value, pad with a zero
if (strlen($hours) == 2) {
  $hours = substr($hours, 0, 1) . '0' . substr($hours, 1, 1);
}

// If the minutes is represented as a single numeric value, pad with a zero
if (strlen($minutes) == 1) {
  $minutes = "0$minutes";
}

// Generate the timezone string
$timezone = "$hours:$minutes";

?>

TinyLink, Ajaxified!

I made use of TinyLink's JSON API.

I put the following links at the bottom of my posts:

<p class="post-tinylink"><a href="http://tinylink.co.za/create.php?url=http://blog.charlvn.za.net/2007/10/tinylink-api-consumer-php-shell-script.html">Get TinyLink</a></p>

The url parameter is basically just the permalink to the post.

Then I call the following JavaScript after all the links have been loaded:

var tinylink = Array();
var tinylinkA = document.getElementsByTagName("a");
var tinylinkExpected = 0;
function tinylinkDisplay() {
  tinylink.forEach(function(element, index) {
    if (element != null) {
      var textnode = document.createTextNode(" " + element["url"]);
      tinylinkA[index].parentNode.insertBefore(textnode, tinylinkA[index].nextSibling);
      tinylinkExpected--;
      tinylink[index] = null;
    }
  });
  if (tinylinkExpected > 0) {
    window.setTimeout(tinylinkDisplay, 200);
  }
}
function tinylinkFetch(event, count) {
  event.currentTarget.removeEventListener("click", tinylinkFetch, false);
  event.preventDefault();
  var href = event.currentTarget.href.replace("create.php", "api/json.php");
  var i = 0;
  while (i < tinylinkA.length && tinylinkA[i] != event.currentTarget) i++;
  href += "&name=tinylink[" + i + "]";
  var script = document.createElement("script");
  script.setAttribute("type", "application/javascript");
  script.setAttribute("src", href);
  document.getElementsByTagName("body")[0].appendChild(script);
  tinylinkExpected++;
  tinylinkDisplay();
}
for (var i = 0; i < tinylinkA.length; i++) {
  if (tinylinkA[i].href.indexOf("http://tinylink.co.za/create.php?url=") == 0) {
    tinylinkA[i].addEventListener("click", tinylinkFetch, false);
  }
}

Ok, I realise this still needs a lot of work, but at least it functions in Gecko. What needs to be done:

Test in other browsers.
Add comments.

This script is built on the concept of "ideal" semantics where the content and behaviour is totally separated. This is also a good example of graceful degradation.

Anyway, now I go to bed. Expect improvements to this script to be published sometime into the future whenever I get time (not like I really had time now but anyway).

Tinylink API: Consumer PHP Shell Script

Here is a little script I wrote that consumes the Tinylink REST API.

#!/usr/bin/php
<?php
error_reporting(E_NONE);
$error = false;
foreach ($_SERVER['argv'] as $i => $uri) {
  if ($i) {
    $tinylink = file('http://tinylink.co.za/api/rest.php?url=' . urlencode($uri));
    if ($tinylink) {
      echo "$uri = $tinylink[0]\n";
    } else {
      echo "Error getting Tinylink for $uri\n";
      $error = true;
    }
  }
}
if ($error) {
  exit(1);
}

I can't figure out how to return a "1" to the calling process when there's an error. Any suggestions would be helpful.

I'm switching off error reporting so that the following output doesn't simply appear while you're piping the output from this script into some other script:

Warning: file(http://tinylink.co.za/api/rest.php?url=charlvn.za.net): failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found
 in /home/charlvn/temp/tinylink.php on line 6

Oh yes, and you probably wanted usage instructions. Here are some examples:

./tinylink charlvn.za.net
./tinylink justinhartman.com stii.za.net charlvn.za.net

This assuming that you save the above file to somewhere as tinylink, that your CWD is the directory containing the script, and that you made the script executable (like with chmod +x tinylink). Oh yes and naturally you must have the PHP CLI installed and it's at /usr/bin/php (otherwise you'll just have to adjust the script as necessary).

This is a quick hack so not really a "proper" shell script but if you manage to come up with any improvements please comment and let us know! Thanks!

Update: Thanks to Peter Rotich and H* for helping me right seconds after posting. The answer was blatantly obvious: use exit(1)! Please see the updated script.

Teh First Law of Pwnage

I dunno how many ppl enjoy pwning noobs as much as me, but I strictly live by teh following code:

Teh First Law of Pwnage states taht If a noob can get sk3w13d, he will get sk3w13d in teh shortest possible amount of time.

So watch out Cape Town noobs! Teh uber-leet pwnerers are coming from George heading ur way for teh 27 Dinner!!! My posse and I will be arriving on ur shorez soon enuff...

So far Belinda, Bev, Jayx and Stii (and of course I lol) iz all coming wiff!

Updatez: Will be bringing lotz of LARTs wiff us such as teh infamous "Clue x 4" (modified 2 x 4 plank specially designed for noob pwning).

Lithium, Batteries & Recycling

The last couple of weeks, Energizer has been advertising their Lithium batteries on South African TV.

Lithium batteries (especially Lithium-ion polymer batteries) are pretty awesome (high voltage, long life, easily rechargeable, lightweight, etc.). They also seem to be generally considered as environmentally friendly, although I did find the following:

Lithium (metal) batteries contain no toxic metals, however, there is the possibility of fire if the metallic lithium is exposed to moisture while the cells are corroding. Most lithium batteries are non-rechargeable and are used in cameras, hearing aids and defense applications. For proper disposal, the batteries must first be fully discharged to consume the metallic lithium content.

I am quite concerned about recycling of batteries (actually, recycling in general). Some batteries contain harmful chemicals. For example:

Modern batteries are often promoted on their environmental qualities. lithium-based batteries fall into this category. While nickel-cadmium presents an environmental problem on careless disposal, this chemistry continues to hold an important position among rechargeable batteries. Power tools are almost exclusively powered by nickel-cadmium. Lead-acid batteries continue to service designated market niches and these batteries also need to be disposed of in a proper manner. lithium-ion would simply be too fragile to replace many of these older, but environmentally unfriendly, battery chemistries.

Check out the page on Recycling batteries at the Battery University - definitely worth a read in my opinion. However, I think even Lithium needs to be recycled in any form. For example, you probably don't want it in your normal drinking water unless absolutely necessary. See the Lithium pharmacology Wikipedia article for medical uses. Some Lithium salts affect the mind.

My frustration is with the state of recycling in South Africa. We have some initiatives in the City of George, but are people really aware of these issues? If you ask me, we need to do something about the situation. We are talking about the future here; we are blessed with a beautiful country and we definitely don't want to mess it up by polluting our environment.

Muti.co.za & Censoring

There has been a lot of posts the last few days again about censorship on Muti.

One of the things I personally like about Muti is that the signal-to-noise ratio is much better than the "big" international sites. I get notified via Jabber and IRC every time somebody posts a link. Because of the multiple syndication methods, I spend time looking at each link. I don't mind browsing through links to quality content (although I might not read all of the actual content myself) because at least it isn't a waste of my time. This counts even if the link is posted by the author. However, I do not appreciate the author posting a link to every single post he writes. Some of the posts on my blog might be interesting, but most of them do not deserve to be posted to Muti. Muti is about the crem-de-la-crem to me. If I find a blog through Muti (or whichever means) and I really like it, I'll subscribe to the blog's individual Atom or RSS feed to stay current with all the posts. Syndication sites such as Afrigator and Amatomu (in alphabetical order) are there for syndicating all the posts. If every author posted all his own stuff, what would make Muti different than these?

On the bigger sites, there are thousands of posts every single day. Nobody wants to look through all of them; individuals normally only follow specific categories, tags, or a combination of them and vote up whatever they like. Muti only gets a couple of dozen links per day, so posting a link is considered by many to be much more "major" than on the other sites.

By this I am not accusing anybody of anything. I would just really like to see responsible use of such a great online social website.

Update: Vincent Maher wrote a very good objective post about this issue that is very-much worth reading in my opinion. I also commented just now. I do think that his reply that my comment on Richard's post was blatantly false is a little unjustified though.

Milkshakes

After studying, you really need a reward for all the hard work to keep you motivated. Therefore, I hack for a few hours and drink a nice home-made milkshake before going to bed.

For a strawberry milkshake, here is my recipe:

Milk
Cream
Strawberries
Honey / Syrup
Cream Cheese

Like with all my other recipes, this is simple. Throw in as much of each ingredient as you like. Yes, all together, and blend. No steps. Experiment a little.

Rather nice, but be careful with the cream and cream cheese as they can make it really rich.

My favourite is still Banana though. Chop almonds as finely as possible in water. Then throw Bananas in. You can add natural vanilla, carob powder, honey, etc. No dairy products - good for the allergies this terrible George air is giving me! Does this still qualify as a "milkshake" though?

Breakfast: Yogurt, ja?

They say that you can't study on an empty stomach. Therefore I invented my own yogurt "recipe" (not really much of a recipe, but who ever said I'm not allowed to show off a little). In the order you make it:

Woolworths Greek yogurt
Thick cream
Honey
Finely chopped Peacan nuts

First take a bowl (the one you want to eat from, naturally) and throw as much yogurt as you like into that. Note that I'm not affiliated with Woolworths but they do (usually) have good quality food so obviously it's not too bad. Why I like their Greek yogurt in particular is that it's delicious and very very thick (as in not thin, don't worry you won't get stupid from eating this). It feels almost like eating cheesecake.

Thick cream to compliment the thick yogurt is the natural choice. Mix the two thoroughly or leave the cream drifting on top.

Honey - yes tasted nicer and is much healthier than syrup. Just make sure you buy South African honey (organic and high quality), not that Chinese shit full of poison. I would buy electronics from there any day but not food. I think they need to ban Chinese food imports actually, until that country starts to shape up their health policies.

Peacan nuts - yeah chop these finely in a blender (if you like) and these will make it really nice. Throw it on top. Don't spoil the smoothness of the yogurt by mixing. Throw the honey on the yogurt first, don't throw the honey on top of the Peacan nuts because it won't stick properly and become a mess. Let the honey stick on top of the yogurt and then let the nuts stick on top of the honey and yogurt mess.

This is much nicer to me than mixing it as it keeps tasting different all the time. Some bits have more honey, cream, etc than others.

Ok, my limited views on making a nice breakfast or brunch... Even though you might take brunch at 16:00 in the afternoon because you only woke up at around lunchtime because you hacked until dawn yet again and the whole family is angry at you for being a weirdo geek. (Obviously, I have been there...)

Aphophysis

twobombs created some really cool artwork using Aphophysis. These are my favourites:

Well done, oubie! You rock!

Now, when are they producing a Linux version? :)

Main

Latest

Archives