Author Archives: admin

Downloading from Soundcloud

A couple years ago, downloading a song from Soundcloud used to be pretty trivial. Their server would send you the complete 128kbps MP3, and then the local embedded control would allow you to seek at will. Because the file arrived in one large chunk, it was both easy to identify in cache, and easy to copy somewhere else to play back. Sometimes this still works… You’ll know by looking at the dev console, and see if it shows a huge MP3 file transfer. If so, you’re in luck! Copy it from the cache and you’re set.

Evidently they’ve changed this practice for other tunes, possibly to improve the latency of seeking at random in tracks, or possibly because they don’t want people getting music they shouldn’t be able to get. You can get Greasemonkey scripts which put the download button back, but these simply fire the URL off to an third-party site which “somehow” reconstructs the song and then sends it back your way. Very black-box magic stuff indeed.

However – If you can stream it, you can download it, as they say. Let’s take a look at how Soundcloud actually gets a song to you, and see if we can still figure out how to download something we may not really be allowed to.

Start the browser’s Developer Console and then browse to a song you want to hear. Keep an eye on the “network” activity, it will give you clues as to what is actually going on. As the song begins playing, you’ll see a lot of small network requests to magically named files:
mini_files

This seems promising. Download one or two and run “file” on it, and you get:
$ file *
c5f47vUnF3Ow.128.mp3?f10880d39085a94a0418a7e163b03d5226edfe2317e6aa1445547d76cf23a7ca5b08b0b9169eed2c0a13f681ab93c51d8e788dcaa887622ee2905d7463e4fd982e918b5b687caf75047026a3429731c5010a16: MPEG ADTS, layer III, v1, 128 kbps, 44.1 kHz, JntStereo
c5f47vUnF3Ow.128.mp3?f10880d39085a94a0418a7e167b03d5249919aaf544816306c9a5e3ca05a129454accfdda2750c51705ac2f68f036a37b2c482058312ab10625db87a6e3ab6dc1d1631dbd883a3f38786db484e66359daf667314eb8f03: MPEG ADTS, layer III, v1, 128 kbps, 44.1 kHz, JntStereo

Okay, so Soundcloud has broken the file into parts and is playing them back in sequence. You can pop one of these into your media player and listen to a portion of the song. We’re close, but how do we know where to find all these parts and put them together in order? Easy: there’s an m3u8 that has that for you – check the Dev Console again! Soundcloud’s player is using this to fetch the data in order from various URLs, and then stream it to you. For example, something like this:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:1
#EXTINF:1.985272,
https://ec-hls-media.soundcloud.com/media/0/31762/c5f47vUnF3Ow.128.mp3?f10880d39085a94a0418a7e163b03d5226edfe2317e6aa1445547d76cf23a6c6277ac4a31c0a01842ddc2964536e126c07aa07555cce1108baa06b42a8a56548b485330689f4477467fbe84e3701184db02ae8
#EXTINF:2.977908,
https://ec-hls-media.soundcloud.com/media/31763/79410/c5f47vUnF3Ow.128.mp3?f10880d39085a94a0418a7e167b03d5249919aaf544816306c9a5e3ca05a13988480bef0d85fac22fe42482ceefe72b45f93798b344c894177b5503fecfe001728ea53a521df650649905f1f820170622633f8408d974a
#EXTINF:4.989302,
https://ec-hls-media.soundcloud.com/media/79411/159240//c5f47vUnF3Ow.128.mp3?f10880d39085a94a0418a7e168b03d52f1af9dc5c031765e833721190ff39af5838b01fab4994aa8e1cbf7325eac7c17bbc32c9af8a4f995094f28154eaf2285d51dae2371e29d5df21535551f28bc9d3d6cdf11c7f30fd3
#EXTINF:9.978604,
https://ec-hls-media.soundcloud.com/media/159241/318900/c5f47vUnF3Ow.128.mp3?f10880d39085a94a0418a7e169b03d52a8b3c33b59894223455854e2f950e750c62859d445307c2908ba5ad3f2b1c73b802a1616248b94bd6c5babf25058d658ad1566709910633c2cd63c67371fbb6692e6db87e986a283f0
...
#EXTINF:9.978604,
https://ec-hls-media.soundcloud.com/media/3352449/3512109/c5f47vUnF3Ow.128.mp3?f10880d39085a94a0418a7e061b03d52eb7d518bdc510e28891131df942c66df2d11c65c5f6f65b400607098a9a36cb891691d16355d126d2ecbc6451bb48f82561d15b81d006b4104caff07fffa8361c42652b9341faefa8f4513
#EXTINF:5.642352,
https://ec-hls-media.soundcloud.com/media/3512110/3602388/c5f47vUnF3Ow.128.mp3?f10880d39085a94a0418a7e061b03d52eb7d518bdc510e28891131df942c66df2d11c65c5f6f65b400607098a9a36cb891691d16355d126d28cfc6401ebd8f8214c9a4dd73e90391e79897cc892a72bcd4883580fdef67b3cf0e33
#EXT-X-ENDLIST

An interesting aside, it seems these URLs time out after a short period of time, leading to 403 Forbidden errors if you try to access it again. No doubt these huge URL parameters point to some browser session or timestamp which becomes invalid after a while. If that happens, reload the page and start playing again to generate new files.

So to recap all this: we need to

  • take the m3u8 file,
  • retrieve each mp3 segment,
  • and concatenate them together.

Getting the m3u8 programmatically is hard, so just copy it from the browser : ) And to put these together you’ll need mp3cat installed – see http://tomclegg.ca/mp3cat for info.

#!/usr/bin/env perl
open(FP,"playlist.m3u8") or die "can't open playlist: $!";

my $piece = 0;
while( < FP > )
{
chomp;
next if ($_ =~ m/^#/);
$filename = sprintf('outdir/%02s.mp3',$piece);
print `wget --no-check-certificate -O $filename $_`;
$piece++;
}

print `cat outdir/*.mp3 | mp3cat - - > output.mp3`;

Arduino Raytracer

Every programmer worth his salt has written a raytracer of some sort. It’s one of the classic “recursion exercises”, with a fair amount of math to wrap your head around. I’m not interested in writing one to run on the PC – there are already far more capable ones that can run on pretty much any hardware, complete with optimizations and features I couldn’t hope to implement in my lifetime.

Instead, I decided to write a raytracer which targets the Atmega 328 microcontroller in my Arduino! In the end, I succeeded in banging one together that calculates ray-triangle intersection, and each triangle can have a material attached with ambient / diffuse / transparent / reflective values and an RGB color. The scene is compiled in with a .h file (stored in PROGMEM), and outputs a PPM image pixel-by-pixel over the serial port.

The same engine can be used on the PC with a few modifications, so I was able to benchmark the performance of the Arduino in comparison with a modern laptop. The “detailed scene” rendered a 640×640 image in 263 seconds on the laptop. The same scene, scaled to only 64×64, still took 4008.471 seconds… in other words, the PC outperformed the Arduino by a factor of ~1500.

I posted the code, and further write-up, on the Arduino.cc message board at this link: http://forum.arduino.cc/index.php?topic=281076.0. There are a lot of features that COULD be added here, but none that I actually intend to do. If I need a raytracer in the future, I’m just downloading POVRay : )

pfSense on Firebox X500

IMG_4881 Previously I had run pfSense as my firewall / router on a discarded Celeron 700mhz machine, with a lot of spare NICs in the PCI slots. Browsing around the local Goodwill and I stumbled upon this Watchguard Firebox X500 for $10. Finally, real rackmount gear for my 2×4 server rack!

I’m sure whatever software comes on this thing is great and wonderful, but I didn’t even try to use it. First step if you have one of these – hop on over to the pfSense Firebox wiki page and do some research. Popped open the case, swapped in a 2gb CF card with the pfSense no-VGA embedded image, and set to configuring it. The most difficult part of all this was getting a USB to Serial cable that worked… and then a Null Modem cable… and then having to do the configuration in a virtualized Windows XP machine because there are no OSX drivers for the WCH341 chip in my cheap eBay usb-serial converter.

Now that it’s all set up I can easily access via ssh or http, so serial access isn’t as important. The hardware in this is just a 1.2ghz Celeron, with 10/100 Ethernet ports and some 256mb RAM. Obsolete, yes, but that red paint job never goes out of style.

Wacom ArtPad II (KT-0405-R) to USB

1119141209 (1) Six or so years ago, I picked up a Wacom ArtPad II digitizer from Freecycle. It came with serial cable and power brick, but no pen. These devices aren’t like an iPad or whatever (which use a capacitative touchscreen) where you can use your finger or a cheap stylus. Wacom pens have built-in smarts to communicate with the tablet and relay pressure information, pen/eraser mode, button click, etc. Without the pen, it’s downright useless. A few eBay searches revealed that pens were crazy expensive to replace at the time. Into the spare bin it went.

Recently I had an interest in getting this working again. I did more research and found that Wacom pens are actually somewhat interchangeable – in this case I can use any “Penabled” pen on it, and it’ll probably work. The new Samsung Galaxy Note S tablet uses Wacom styluses and the replacements are cheap – $8 on eBay, so I ordered one. Holding the pen over the tablet causes the LED to switch from orange to green, which was good enough for me to figure it’s working.

New problem – I have no machines with a serial port any more, and even if I did, Wacom drivers don’t support serial tablets any more. The solution comes from the WaxBee project, which uses a Teensy 2.0 to sniff old-protocol tablet packets and re-encode them to emulate an Intuos2 (a modern USB tablet with driver support). Great! Ordered a Teensy and cracked open the tablet to set to work.

First, I knocked off the serial connector and voltage regulator (soldered across that instead), as the USB will provide the +5V needed to drive the tablet. Next step is removing the ADM202 TTL-to-RS232 chip, where things took a disastrous turn: my overeager Dremel application removed the chip as well as the pads it sat on, and gouged out some of the PCB to boot. Crud, now I have to follow traces to figure out where it was supposed to go. Took a couple high res photos and gave up for the night. (If you need more info – circuit board photos, etc – check out this series of posts: http://forum.bongofish.co.uk/index.php?topic=2088.msg20757#msg20757)

test The actual solution turned out not so bad: had to lift three pins on the Motorola chip and solder those to the Teensy. The hid_debug tool from pjrc (and associated WaxBee firmware) was invaluable in figuring out when I had the wires going the right way. Once that was done, superglue secured the wires and electrical tape held the Teensy. I cut a chunk off the side of the enclosure for the USB micro-port to fit, and drilled a hole for the Teensy button if I ever need to reprogram it. Covered with electrical tape, plugged it, and miraculously it all worked!

Crazy Taxi Garmin GPS Voice

Here is a project I’d been talking about for years, but never actually pulled it off until recently. It’s a voice set for a Garmin GPS, which replaces the default “narrator” with the obnoxious announcer from Crazy Taxi.
All voice clips were pulled from the PC version of the game, which had them in convenient .wav format already. Some trim / normalize with Audacity and import into Garmin Voice Studio to assemble together. I wanted to do more than just the announcer, but didn’t have any luck with that… there just weren’t enough clips to make it happen.

Here’s a video of the system in action:

I hosted all the downloadables on archive.org. You can get it from here:
https://archive.org/details/Garmin-CrazyTaxiVoice

cfg_parse on Sourceforge

cfg_parse Icon cfg_parse, that tiny config parser I wrote a few months back, has now been moved to Sourceforge. I also took the time to make some Doxygen comments and generate a manual for it. No major bugs were found but a couple minor tweaks help tighten up the release.

Check it out here: http://cfg-parse.sourceforge.net

The easiest, easiest way to use this is to add it as a subversion externals for your project – that way it will automatically get checked out with the rest of your code, and you can build it right in! The syntax would be something like…
svn:externals svn://svn.code.sf.net/p/cfg-parse/code/trunk@2 cfg_parse

Teeny Tiny MIDI Controller

I’ve been working sporadically on this tiny MIDI controller.  It targets the smallest PIC microcontroller available with onboard A/D converter – the PIC10F220.

Currently it reads two potentiometers and a button input using three of the GPIO pins, and it bitbangs the remaining output pin to produce MIDI-Out at 31250hz using cycle-counted delays.  Since practically everything is already on-chip, this thing could just be epoxied straight to the enclosure.  There are only two or three more components – no PCB required!

schemThis microcontroller sports a whopping 256 instructions and 16 bytes of RAM.  Even so, the MIDI controller code consumes less than half the available resources after a modest amount of optimization for size.  One might see this as my response to using a certain 32k ROM / 2k RAM microcontroller to blink some lights, but I won’t admit to being that petty.

mplabWhat to do with the remaining space?  An easter egg isn’t a bad idea…  Hold the button down on boot, and the controller will dump “HTTP://WWW.HACKADAY.COM” as a series of MIDI note-on messages.

Unfortunately I haven’t built the thing yet, but I did simulate it.  Here’s the logic analyzer input, showing MIDI messages being sent (delays removed):

plot_editYou can listen to the result here: EasterEgg.mid

Code follows.

;
;
; Easter Egg: hold button on startup ;-)
    btfsc    GPIO,GP3    ; 2-3
    goto skip_egg

; "note on" running mode
    movlw    (0x90 | midi_channel)
    movwf output
    call serial_bitbang

; reset pointer
    clrf    egg_timer

; "HTTP://WWW.HACKADAY.COM" -> "48 54 54 50 3a 2f 2f 57 57 57 2e 48 41 43 4b 41 44 41 59 2e 43 4f 4d"
egg_loop:
; load ptr into w reg
    movfw egg_timer
; go get the next ascii value
    call egg_table

; done?
    iorlw    0x00
    btfsc STATUS,Z
    goto skip_egg

; send note (on)
    movwf output
    call serial_bitbang
; send note volume
    movlw 0x7F
    movwf output
    call serial_bitbang

; delay
    movlw 0x7F
    movwf output
egg_delay_outer:
    movlw 0x7F
    movwf temp
egg_delay_inner:
    nop
    nop
    nop
    nop
    decf    temp
    btfss    STATUS,Z
    goto egg_delay_inner
    decf    output
    btfss STATUS,Z
    goto egg_delay_outer

; send note (off) - again
    movlw egg_timer
    call egg_table
    movwf output
    call serial_bitbang
; send note volume 0x00 == off
    clrf output
    call serial_bitbang

; next ASCII char
    incf egg_timer,f

    goto egg_loop

skip_egg:
; Send the first MIDI CC message.
; ...

; ...
; rest of project code...
; ...

; lookup table for hackaday URL
egg_table:
    addwf    PCL,F    ;add offset to pc to
                    ;generate a computed goto
    retlw    0x48
    retlw    0x54
    retlw    0x54
    retlw    0x50
    retlw    0x3a
    retlw    0x2f
    retlw    0x2f
    retlw    0x57
    retlw    0x57
    retlw    0x57
    retlw    0x2e
    retlw    0x48
    retlw    0x41
    retlw    0x43
    retlw    0x4b
    retlw    0x41
    retlw    0x44
    retlw    0x41
    retlw    0x59
    retlw    0x2e
    retlw    0x43
    retlw    0x4f
    retlw    0x4d
    retlw    0x00
    END

GregerQuest

gqTitle
Before there was World of Warcraft, there was Everquest. I never played it but I had a buddy in college who did: he talked about how amazing the MMO genre was, with all its social aspects, technical quirks, and just plain cool experiences.

Naturally it inspired me to make my own clone. In Visual Basic. With 2d graphics and piping everything through the Winsock .ocx control. Full of programmer art made with a pirated copy of Bryce 3d.

gqChars

Ah, the wide-eyed days of a young programmer in a rapidly expanding genre. It’s like I was playing out every cliche of the Gamedev.net newbie at once. The one thing I had going for me, though, was a thorough understanding of the impossibility of such a project – and a sense of humor about it all. For example, I often joked that the only enemies were going to be Gelatinous Cubes, because that would be easy to render. I once wrapped a photo of my roommate’s face on a sphere to use for a hideous player character head. And so on. Eventually I got about as far as a character select screen, with a couple songs, before giving up and moving on to something else.

Unlike SlugFest, this is one that isn’t ever going to get off the ground. If anyone is interested in the art and music resources, you can have them: hereby released into the Public Domain.

shore

This isn’t my only brush with the MMO genre. Later on some friends and I tried another take on it (“Draconis”) as the U of A Game Dev club project – with a result somewhat similar to The Mana World, and it once held up to five players online simultaneously. The real killer for these kind of games is the sheer amount of content required. Though Draconis worked technically, all the content in the world was viewable at first login. Hardly a compelling MMO experience.

Download GregerQuest Resources: .ZIP file, 1.9mb.

Smoke Help (for OSX)

This is a tool to help people quit smoking. My wife gave me the idea, I just wrote it.

coach

Start the program up and put it into Learn Mode. Every time you have a cigarette, hit the “SMOKE” button. This trains the program, over the course of 100 cigarettes, your (natural) average time between cigarettes.

Then put it into Coach Mode. At this point the program will begin telling you when it’s time to smoke again, using an alarm. Don’t smoke until instructed to do so, and then hit the button when you do. The program starts by alarming at the learned rate, but extends the interval each cigarette. Eventually your rate will slow down to the point where quitting is natural.

At least, in theory. My wife quit without my help so I wasn’t ever able to test it.

This is the sort of thing that would be super-ideal as a mobile app. I don’t have time nor inclination to set that up, but I’d love to see it. So, this is now public domain software. (Included font isn’t though).

Smoke Help OSX Binary – .zip – 2.45 MB
Smoke Help source code – .tar.gz – 227kb

Minus Infection

I took a programming languages class in college, which used Scheme to demonstrate concepts like lexers, parsers, etc. The final project was to write our own interpreted object-oriented language. I enjoyed the class but never quite got the hang of Scheme. So when the time came to write the language, I did the minimum possible in Scheme, and implemented the rest as a standard library add-on to my own language.

Cutting things to the bare minimum resulted in a language with only seven constructs, and the only arithmetic operation was subtraction. Addition would thus be implemented as minus (a, minus(0, b)). IF statements were simply WHILE loops that ran zero or one times. And so on.

Sadly, I don’t have the source code any more. But I do have the README file.

Minus Infection
A Programming Language by
Greg Kennedy

I. Introduction
What is Minus Infection? MI is a programming language which aims to be Turing complete and support object-oriented programming with as few primitives as possible. It is written in Scheme. Functionality is provided by external Library files, which define many of the functions you would otherwise miss from your favorite language.
Because of this, the actual implementation of MI is rather simple, but powerful enough that you can use the built-in functions to build vastly more complicated ones.

But why? Scheme is a fine language. It is really quite powerful. But I just can’t get the hang of all those parentheses and cars and cdrs. So I thought I’d minimize the work I had to do in Scheme, and instead move that work into MI. It’s also sort of an experiment for me to discover just how little is really needed to make a usable language.

II. How do I use it?
MI source files are simply text files – create them in Notepad or vi or whatever.
Your source file is the main routine of your program.

III. What are the primitives?
MI recognizes as “true” any expression which evaluates to less than 0, and “false” as any expression which evaluates to “equal to or greater than 0” (i.e. “not false).

The only variable type MI understands is integer arrays. You can create single variables in your source and omit the subscript when later referencing them – the parser will simply fill in the missing subscript with a [0].

See section 4 for methods for dealing with strings. Internally they are represented (like in C) as an array of integers representing character codes.

Arrays maintain their own sizes and will warn of an out-of-bounds access. For strings, no special termination character is needed.

These primitives are built-in to MI:
MINUS (X, Y) – perform the calculation X – Y and return the result.
LTZ (X) – returns -1 if X is less than 0, 0 otherwise.
WHILE (COND, BLOCK) – continually executes BLOCK as long as COND is true.
SET (X, EXP) – sets X to the evaluated result of expression EXP.
DEFINE (NAME, PLIST, BLOCK) – sets
INT (NAME, SIZE) – declares an array named NAME of length SIZE
DOT (NAME, MEMBER) – OO operator: call member function or access member variable
PRINT (x) – Prints X to the screen. X is a string.

IV. What is in the libraries?
You’ve no doubt noticed that the language seems a little sparse. That’s because all the real functionality comes from the libraries.
To load a library, simply place this command in your source file:
LIB “FILENAME” – when parsing, this line causes the execution of FILENAME.

The standard library is called default.lib. It provides the following functions:
< (X, Y), > (X, Y), = (X, Y),
<= (X,Y), >= (X, Y) – Conditionals returning true if X (op) Y.
IF (COND, BLOCK1, BLOCK2) – if COND is TRUE, executes BLOCK1; else executes BLOCK2
– (X, Y) – behaves like MINUS
+ (X, Y) – calculates X + Y, returns result
*, /, % (X,Y) – multiply, divide, modulus
NAND(X,Y), NOR(X,Y), NOT(X),
AND(X,Y), OR(X,Y), XOR(X,Y) – boolean operations.
FOR(X, COND, INC, BLOCK) – Similar to C for loops
STRCPY (X, Y) – copies string Y over string X.

V. Example code?
This program prints “Hello, World!” to the screen.

LIB "default.lib"
VAR (HWORLD, 14);
STRCPY(HWORLD,"Hello, World!");
PRINT(HWORLD);