0dbfs.pl: Bringing a Nuke to the Loudness War

Ever heard of the Loudness War? It’s the steady ramp-up in perceived volume that has plagued popular music for the past 30+ years. Producers have steadily chipped away at dynamic range in an effort to make a hotter track. Audio signal-to-noise ratio suffers. Nowadays the typical radio song has a 3-6 dB dynamic range (for comparison, CDs have a theoretical 96 dB range).

Well folks: I’m playing to WIN. Here is a compressor that scales ALL of your peaks to 100%: in other words, compression at a ratio of INFINITY : 1… or a dynamic range of 0. TAKE THAT, PRODUCERS.

Here’s an example in action, cutting from Clair de Lune. First, the original. Then, the RADIO-READY version.

Doesn’t this just sound so much more… hip / edgy / loud / etc?

Interestingly, I tried it on a selection of pop music. It makes for a great demonstration of “listener fatigue”.

0dbfs.zip – zip file containing Perl script, 1.7 kB

cfg_parse – Simple Config File Parser in C

There is no standard way in C to parse config files. Several libraries have sprung up to solve the problem, but perhaps that is overkill for what should be a simple operation.

Presenting: cfg_parse – a compile-in solution for reading key-value pairs from file(s), looking up settings, and writing a config file back to disk. I could write more about it, but the included main.c shows all the functionality.

// driver test program for cfg_parse

#include "cfg_parse.h"

#include

int main(int argc, char **argv)
{
// Pointer to a cfg_struct structure
struct cfg_struct *cfg;

// Initialize config struct
cfg = cfg_init();

// Specifying some defaults
cfg_set(cfg,"KEY","VALUE");
cfg_set(cfg,"KEY_A","DEFAULT_VALUE_A");

// "Required" file
if (cfg_load(cfg,"config.ini") < 0)
{
fprintf(stderr,"Unable to load cfg.ini\n");
return -1;
}

// Several "optional" files can be added as well
//  Each subsequent call upserts values already in
//  the cfg structure.
cfg_load(cfg,"/usr/local/etc/config.ini");
cfg_load(cfg,"~/.config");

// Retrieve the value for key INFINITY, and print
printf("INFINITY = %s\n",cfg_get(cfg,"INFINITY"));

// Retrieve the value for key "KEY", and print
printf("KEY = %s\n",cfg_get(cfg,"KEY"));

// Delete the key-value pair for "DELETE_ME"
cfg_delete(cfg,"DELETE_ME");

// Dump cfg-struct to disk.
cfg_save(cfg,"config_new.ini");

// All done, clean up.
cfg_free(cfg);

return 0;
}

Download: cfg_parse.1.0.tar (version 1.0, 4kb .tar.gz)

iNES Header Fixer

Auditing ROM collections has been made much simpler over the years thanks to concerted efforts of many cartridge purchasers, dumpers, cataloguers, and coders.  Using a combination of RomCenter (or ClrMamePro) and a No-Intro .Dat file, one can quickly fix an entire collection in one go: repairing names, verifying regions, checking for problem ROMs, producing have-lists, etc etc.

NES roms throw a particular curveball because they carry a 16-byte header with a little info about the board within the cart.  Emulators need this info, but it isn’t verified by No-Intro since it’s not part of any real dump.  As a result iNES headers remain missing, incorrect, or filled with crap data.  A new repair tool is needed.

ines-fix.py is a Python script which can repair defunct iNES headers using an external cart database.  It does this by extracting the data portion from the file, calculating a crc32, looking for the crc the xml file, and then rebuilding a header based on that info.  If the old and new headers differ, the file is overwritten with a fixed version.

Be careful with this tool: it’s sort of a blunt object, and may trash your rom collection if not used carefully.  Make backups.  In particular it writes only iNES v1 headers, expecting emulators to cope with the shortcomings of this format.

Output samples:
-> No change to existing header

CRC check: 4318A2F8
Found CRC match: Barker Bill's Trick Shooting
*** Header unchanged: not writing replacement file.
----------------------------------------------------

-> Adding a battery

CRC check: 1F6EA423
Found CRC match: Baseball Simulator 1.000
*** HEADER UPDATED ***
oldHeader: 4e45531a081010000000000000000000
newHeader: 4e45531a081012000000000000000000
All done.  Wrote new file roms/Baseball Simulator 1.000 (USA).nes

-> Correcting a mapper, vertical mirroring, and removing “DiskDude!”

CRC check: 9BDE3267
Found CRC match: Adventures of Dino Riki
*** HEADER UPDATED ***
oldHeader: 4e45531a0204004469736b4475646521
newHeader: 4e45531a020431000000000000000000
All done.  Wrote new file roms/Adventures of Dino Riki (USA).nes
----------------------------------------------------

-> Standardizing on “horizontal mirroring”

CRC check: 3ECA3DDA
Found CRC match: Bases Loaded 3, Ryne Sandberg Plays
*** HEADER UPDATED ***
oldHeader: 4e45531a101041000000000000000000
newHeader: 4e45531a101040000000000000000000
All done.  Wrote new file roms/Bases Loaded 3 (USA).nes
----------------------------------------------------

-> Adding a missing header

CRC check: 50CCC8ED
Found CRC match: Battleship
*** HEADER UPDATED ***
oldHeader:
newHeader: 4e45531a020430000000000000000000
All done.  Wrote new file roms/Battleship (USA).nes
----------------------------------------------------

Here’s the script:

#!/usr/local/bin/python
"""
Converts NES ROMs to iNES format, applying correct iNES header.
Usage: ines-fix 
Supported infile formats are .nes, .pas (headerless .nes)

ROM data is recognized by CRC32 using BootGod's master XML database
  so make sure you have a local copy
"""

import sys
import struct
from binascii import crc32
from xml.etree import ElementTree

##### String holding location of cart db
cart_xml = "NesCarts (2011-09-10).xml"
# uncomment next line to use Nestopia's DB instead
# cart_xml = "NstDatabase.xml"

# Other required vars
i_fmt = 'p'
blob = None
found = 0
oldHeader = ""


# Parse command-line

if (len(sys.argv) != 2):
    print "Usage: " + sys.argv[0] + " "
    sys.exit(0)

# Open rom database
#print "Attempting to open cart db " + cart_xml
tree = ElementTree.parse(cart_xml)
#print "DB opened!"

# Attempt to open supplied rom file

try:
    with open(sys.argv[1], "rb") as f:
        tag = f.read(4)
        f.seek(0)
        if (tag == "NES\x1A"):
            i_fmt = 'i'
            oldHeader = f.read(16)

#        print "Detected " + i_fmt + " format for input file"


        blob = f.read()

except IOError as (errno, strerror):
    print "Error opening " + sys.argv[1] + ": "
    print "I/O error({0}): {1}".format(errno, strerror)
    sys.exit(2)

if (len(blob) > 0):
    format_crc32 = format(crc32(blob) & 0xFFFFFFFF, '08X')
    print "CRC check: " + format_crc32

    # go look up crc32 in db
    game_list = tree.findall("game")
    for game in game_list:
        cart_list = game.findall("cartridge")
        for cart in cart_list:
            if (cart.attrib.get('crc') == format_crc32):
                found=1
                break
        if (found):
            break


    if (found == 0):
        print sys.argv[1]
        print "*** CART NOT FOUND IN DB"
        print "----------------------------------------------------"
        sys.exit(2)

print "Found CRC match: " + game.attrib.get("name").encode('ascii', 'ignore')
#ElementTree.tostring(game)

# retrieve data from game
board = cart.find("board")
mapper = int(board.attrib.get("mapper"))

prg_size = 0
prg_list = board.findall("prg")
for prg in prg_list:
    prg_size = prg_size + int(prg.attrib.get("size") [:-1])

chr_size = 0
chr_list = board.findall("chr")
for chr in chr_list:
    chr_size = chr_size + int(chr.attrib.get("size") [:-1])


battery = 0
wram_list = board.findall("wram")
for wram in wram_list:
    if (wram.attrib.get("battery") is not None):
        battery = int(wram.attrib.get("battery"))

chip_list = board.findall("chip")
for chip in chip_list:
    if (chip.attrib.get("battery") is not None):
        battery = int(chip.attrib.get("battery"))


mirror_4 = 0
mirror_v = 0
pad = board.find("pad")
if (format_crc32 == "CD50A092" or \
    format_crc32 == "EC968C51" or \
    format_crc32 == "404B2E8B"):
    mirror_4 = 1
elif (pad is not None):
# the "h" pad means "v" mirror
    mirror_v = int(pad.attrib.get("h"))


mapper_lo = mapper & 0x0F
mapper_hi = mapper & 0xF0

newHeader = "NES\x1a" + \
              struct.pack("BBBB", \
                ( prg_size / 16 ), \
                ( chr_size / 8 ), \
                (mapper_lo << 4) + (mirror_4 << 3) + (battery << 1) + (mirror_v)
, \
                (mapper_hi) ) + ( '\0' * 8 )

if (newHeader != oldHeader):
    print "*** HEADER UPDATED ***\noldHeader: " + oldHeader.encode('hex')
    print "newHeader: " + newHeader.encode('hex')

    # write new file
    try:
        with open(sys.argv[1], "wb") as f:
            f.write( newHeader )
            f.write( blob )


    except IOError as (errno, strerror):
        print "Error opening " + sys.argv[1]
        print "I/O error({0}): {1}".format(errno, strerror)
        sys.exit(2)

    print "All done.  Wrote new file " + sys.argv[1]
else:
    print "*** Header unchanged: not writing replacement file."
print "----------------------------------------------------"

For those having problems with copy-paste, you can download a .zip containing the script here:
ines-fix

Timewave v1.2

Introduction
Timewave is a 2-d space shooter written in C++ using some of the freely available SDL libraries. It runs on Windows, Linux, and Mac OSX (10.4 or later). The object of this game is to fight through the mass of enemies in each level, defeat the level boss, and finally destroy the (well-armed) enemy space station far from Earth.

It has an interesting “twist” in which you can alter the flow of time. Build up your time meter by playing the game in fast-forward. Playing in slow motion affords you some extra maneuvering room but depletes the meter. If you achieve a high score, you can choose a color flag to represent your ship and have yourself recorded on the high score table.

This game is ‘freeware’ – you can download it and copy it around to anyone you like, you can even alter the graphics to better suit your tastes if you wish, but don’t pull my name off it and try to pass it off as your own work. You can’t get the source code though because it’s a total wreck. I may go and try cleaning it up later for a release, but until then, the binaries in the download should work just fine.

Screenshots
Last updated Aug. 7, 2007 – click for full-size

Recent Changelog

  • Reduction in resource file sizes: some items converted to 256-color, optipng and audio editing shaved further bytes.
  • Renamed hiscore.dat to config.ini, and added more options.
  • New command line options: -f (switch to Fullscreen mode), -c (Create Config: just write config.ini and exit), -w (switch to Windowed mode)
  • Added a “Change Keys” button which allows users to alter the inputs.
  • Added “Arcade Mode” switch to the config.ini file. This causes the game to simulate an arcade machine on Free Play: the title screen rotates with the high score table.
  • The arrow keys now drive the cursor on the high score color picker table.
  • Mac OSX port (PPC / Intel Universal Binary, 10.4 or greater).

Downloads
Last updated May 3, 2012

TI-83 Software

One Christmas in high school, I got a TI-GraphLink, allowing me to connect my TI-83 to my computer and transfer files between them. This was a great tool: now I could save my programs to the PC and no longer be at the mercy of memory resets from battery failure or an upcoming math test.

It also let me try out some Z80 programming, but I didn’t ever get too far with that. Hard-locking the calculator and forcing a battery pull was just too much of a hassle. I followed tutorials and had plans and sample code, but never pulled off a complete program.

All I have left now is just a handful of TI-83 BASIC programs. Sadly my GoldenEye 007 port is gone. I did find a couple of sprite pages for a work-in-progress “North and South” and “Smash Bros.” conversion, both would have been written in assembly and played over the link cable.

There is still “Fruity Vegas Slots”, my most polished game (a slot machine simulator). Also found Hearts (which used cards 2-14 instead of 2-10, J, Q, K, A), Bombs Away (clone of the Palm Pilot game “City Bomber”), a quadratic formula, and my own version of the classic “RACE”.

I deleted a couple of stupid inside-joke type programs and some WIP stuff. Nobody wants to see that : )

I have a lot of fond memories of working on this platform. The slow execution speed and laggy LCD screen reminded me a lot of working on other 8-bit systems, except this one was a handheld and had a wide user base of “everybody in school”. Built-in BASIC is such an endearing feature that I am surprised modern OSes don’t include something like it: QBasic and HyperCard, for instance, were both greatly loved and yet they’re both dead software.

Click here to download the complete package: ti83.tar.gz [2.88 KB]

Duplicate Image Finder (in Perl)

I have duplicate photos in my image library.  We all do.  I want to weed them out.  The trouble with using straight md5 for this, though, is that EXIF data on JPEG files may be altered by my photo management tool – thus they contain the exact same photo data, but the associated extra data (date/time, ICC profile, tags, etc) causes pure checksum comparison to fail.

Here’s a Perl script which iterates through a folder list and sends all files off for md5 digest.  However, any jp(e)g files are first run through jpegtran and saved as a temporary file, so they can be “normalized” (i.e. convert to optimized, progressive, and EXIF-stripped) so the md5 is performed on just image data.  This should find duplicates regardless of image program tampering.

#!/usr/bin/perl -w
use strict;

# path to jpegtran
my $JPEGTRAN_LOC = '/Users/grkenn/Pictures/jpegtran';

# Somewhat Advanced Photo Dupe Finder
# Greg Kennedy 2012

# Identifies duplicate photos by image data:
# strips EXIF info and converts to optimize + progressive
# before performing MD5 on image data

# Requires "jpegtran" application from libjpeg project
# Mac users: http://www.phpied.com/installing-jpegtran-mac-unix-linux/

use File::Find;
use Digest::MD5;

my %fingerprint;

my $ctx = Digest::MD5->new;

sub process
{
  my $filename = $_;

  # file is a directory
  if (-d $filename) { return; }
  # file is an OSX hidden resource fork
  if ($filename =~ m/^\._/) { return; }

  if ($filename =~ m/\.jpe?g$/i) {
    # attempt to use jpegtran to "normalize" jpg files
    if (system("$JPEGTRAN_LOC -copy none -optimize -progressive -outfile /tmp/find_dupe.jpg \"$filename\"")) {
      print STDERR "\tError normalizing file " . $File::Find::name . "\n\n";
    } else {
      $filename = '/tmp/find_dupe.jpg';
    }
  }

  # open file
  open (FP, $filename) or die "Couldn't open $filename (source " . $File::Find::name . "): $!\n";
  binmode(FP);
  # MD5 digest on file
  $ctx->addfile(*FP);
  push (@{$fingerprint{$ctx->digest}}, $File::Find::name);
  close(FP);
}

## Main script
if (scalar @ARGV == 0)
{
  print "Usage: ./find_dupe.pl [ ...]\n";
  print "\tjpegtran MUST be in the path,\n";
  print "\tor edit the script and set JPEGTRAN_LOC to an absolute location\n";
  exit;
}

find(\&process, @ARGV);

print "Duplicates report:\n";

foreach my $md5sum (keys %fingerprint)
{
  if (scalar @{$fingerprint{$md5sum}} > 1)
  {
    print "--------------------\n";
    foreach my $fname (@{$fingerprint{$md5sum}})
    {
      print $fname . "\n";
    }
  }
}

The output looks something like this:

macmini:Pictures grkenn$ ./find_dupe.pl test_lib/
Duplicates report:
--------------------
test_lib/ufo or moon.jpg
test_lib/subdirectory/dupe_7.jpg
--------------------
test_lib/too cool jenny.jpg
test_lib/subdirectory/dupe1.jpg
test_lib/subdirectory/dupe2.jpg

Atari Flashback 2 Mod

The Atari Flashback 2 is a really neat plug-and-play TV game system. It looks like a tiny Atari 2600 and plays 20+ classics for the system. Most interesting to hackers, however, is that the board inside isn’t just a 2600 emulator or recreation – it’s actually a modern, miniaturized 2600-on-a-chip playing real ROM images. Additionally, there is a convenient silkscreened table on the board showing pinouts for adding a 2600 cartridge connector and playing real VCS game cartridges on the system.  Compatibility isn’t 100%, but it’s quite accurate, and the A/V cable is a nice way to hook up to a modern TV (no more fiddling with RF adapters).

Finding a NOS cart connector is tough, but there’s an alternative: source a floppy-to-IDE cable from an old PC – the board connector has the same pin spacing as a cart.  I stuffed cut-down popsicle sticks into the gaps on either side (the connector is wider than a real 2600 cart).

Soldering to the board can be a real pain in the butt, especially if (like me) you don’t have a solder station to do it and are using a blunted Rat Shack iron for the job.  Miraculously, it all worked when I screwed it back together.  Besides the cartridge slot itself I added a few details:

  • Power light
  • Difficulty lights
  • Switch on back to select between “cart” and “built-in games”
  • Removed hardwired A/V cable and replaced with A/V jacks

Cutting the slot posed a new challenge, which I didn’t do a very good job of.  I drilled holes through the cart connector and used long bolts to secure it to the back of the system – carefully spaced so that the would slide into the tabs on the Atari cartridges and release the dust cover.  Then came time to cut a rectangular hole in the top.  I did this… and found I’d put it at the wrong spot.  So I had to cut another hole, leaving a gap around the cartridge and screwing the aesthetic.  This time I got it right and was able to play Star Wars: Empire Strikes Back on my big-screen TV.

Definitely not the most impressive Flashback 2 hack out there, but I’m pleased with the outcome.  Like most of my projects, this one dragged on for a couple of years before reaching a finished state – I’d throw a couple hours at it every few months but never seemed to get it wrapped up.  Now if I could just finish the multicart I’ve also been designing, I could have something to play on it.

Mac Mini Processor Upgrade

Our Mac Mini (Intel, late 2006) is now the family computer.  That’s a big job for this little machine, which last saw an upgrade in 2008 to 2gb RAM.  However, it still poked along on the Core Solo 1.5ghz processor that it shipped with.  That was decent in 2006 but it’s slow in 2012: when your Flash games look like a slideshow, it’s time to upgrade.

Fortunately, the top end processor for this machine (Core2Duo 2.33ghz, the “T7600” model) has been out of production for some time and prices have finally fallen below $100 shipped on eBay.  I snagged one for $91 and went through with the install today.  It’s a little nerve-wracking working in such a tight space, especially since you must pry the case apart using a putty knife.  Also, I had no thermal compound… the nice guys at Luyet Computers donated the .1 gram or so that I needed at no charge (thanks!) and I was up and running shortly.  I also blew out a ton of dust and cleaned up some stickiness that appeared to be ancient Sprite spilled on the computer.

End result: well, it’s way way faster, as to be expected.  I didn’t measure temperatures before and after but (thanks to eliminating dust) it’s actually running quieter than when I took it apart.  For fun I did a before-after Geekbench scores comparison.  Click here to see the Google Docs spreadsheet.

If you have a Mac Mini, I highly highly recommend this upgrade, it’s great for the price (or you can probably do even better getting the T7200 or T7400 versions… but I was enticed by the turbo clock speed).  We should be good for the foreseeable future, until the HDD dies and I put in an SSD instead : )

SlugFest ’97 DX

Kids have big dreams.  Some of them want to grow up to be scientists or astronauts or football stars or President.  When I was in school I wanted to make video games.  So I took programming classes and worked hard on my craft.  I churned out lines of QBasic spaghetti code and, later, migrated to Visual Basic on Windows to do the same thing.  Surrounding much of what I produced was a feeling that I was destined to do something big with whatever I was working on: I was going to make a million dollars off some shareware game, or I would code up a groundbreakingly massive and openended world (and it would all fit on a 1.44MB floppy), or whatever.

Over a summer break in 1997 my cousin Rusty came to spend a week at my family’s house.  I don’t recall exactly how it happened – something to do with playing a lot of Myst, I think – but I managed to convince Rusty and my sister Erin to work on a video game.  We were going to make an awesome fighting game on PC.  It was going to be released on CD – ostensibly because we could put music in the empty space, but most likely because CD-ROM was the hot item of the day.

Our game was called SlugFest ’97.

And so we set to work, with the enthusiasm that only kids have.  For that whole week we invested our time and effort on producing this game.  We each painstakingly drew out MSPaint sprites for our assigned characters.  We coded and built and playtested.  When we weren’t working on it, we were talking about it: how to improve it, how to produce it, how to market it.  We even took time out to make a “The Making Of” video.  And as time grew short we did, in fact, wrap up a version that we were quite happy with.

Then reality set in: we called a local CD mastering shop (this was in the days before CD-R became widespread) and were told that we would be charged $100 to produce our CD.  In hindsight I think the clerk may have been confused about what we were asking and thought we wanted to book studio time.  In any case, we all realized that the dream was simply beyond our financial resources.  Enthusiasm drifted away.  Though we later made an attempt at a sequel (“SlugFest 2000”), it never made it past initial character design before we all lost interest and started playing BattleMasters on the landing at the top of the stairs.

Well.  I cleared out an old folder on my HD recently and ran across both the compiled version of the game, plus the source code.  Unfortunately, it doesn’t run on the most modern Windows version, and is hit-or-miss functional on the rest.  A quick calculation: Fourteen years of coding experience in the intervening time, including running a college game development club for two years… plus a stash of resources including a 2d game framework… a remake should take very little time indeed.  The actual “game logic” is absurdly simple.  The technology to realize the dream is here too – everyone has a CD burner these days.  Yes, I can do this.  I can release SlugFest.  (Since misquoting Steve Jobs is all the rage these days, I’ll throw in an old favorite: “Real Artists Ship”.)

On to the remake.  It actually was very easy.  The entire thing was rewritten from the ground up in C, using SDL as the backend library (plus SDL_Mixer and SDL_Image to provide sound and graphics loading).  I managed to squeeze in a few “DX Mode” features to inject a bit of modernity into the game.  I even cut some sound samples from ancient recordings of us to make fight sound effects.  The result is faster, better, and smaller than the original… though I included that in the installer too, for completeness’ sake.

In fact the most challenging part of all was the music.  When we wrote the game we had no music sources of our own and couldn’t burn anything to test with, but we wrote the game with CD support expecting to just substitute our own tracks in production.  Most playtesting happened to the tune of either No Doubt’s “Tragic Kingdom” CD, or some electronic “Phantom of the Opera” remix CD.  Inspiring, but copyrighted, and not really fitting for the remake.  Instead, I loaded the game up with MIDI files that I or my sisters had written in our school years and used them for background music.  Some of these hadn’t been heard in many years, because they were in a proprietary format that I first had to write a decoder for.  In the end I found 43 tracks worthy of inclusion, mostly without any musicality or rhythm.  Hey, if it worked for Marvel vs. Capcom 2…

And so we come to the release.  There is a web-downloadable installer here, if you want to try it out:

Download SlugFest ’97 DX – Installer – Windows, version 1.01.  1.6 MB
Download SlugFest ’97 DX – ZIP – Mac OSX (Intel 10.5+), version 1.01. 1.8MB

The finishing touch is here, though: run the MIDIs through a MIDI -> WAV conversion tool (I used WinGroove), create a cue sheet, redo the installer to work from CD, and burn a copy.

Well, there you have it.  Childhood dream: accomplished.  Now if I could just figure out a way to market it… : )