{"id":10,"date":"2010-12-16T19:17:04","date_gmt":"2010-12-16T19:17:04","guid":{"rendered":"http:\/\/greg-kennedy.com\/?p=10"},"modified":"2011-08-19T15:37:34","modified_gmt":"2011-08-19T15:37:34","slug":"midi-carver-a-data-carver-for-midi-files","status":"publish","type":"post","link":"https:\/\/greg-kennedy.com\/wordpress\/2010\/12\/16\/midi-carver-a-data-carver-for-midi-files\/","title":{"rendered":"midi-carver: A Data Carver for MIDI Files"},"content":{"rendered":"<p><strong>Overview<\/strong><\/p>\n<p><span style=\"color: #000000;\">MIDI-Carver is a tool for extracting (partial or complete) MIDI files from a binary block of data.\u00a0 It can be used against a solid data file &#8211; e.g. an ISO image &#8211; to attempt to retrieve MIDI data\u00a0into standalone files.\u00a0 It also can parse images of damaged filesystems and makes some attempt at recovery even when the source has damaged one or more tracks, or obliterated the MIDI header.<\/span><\/p>\n<p><span style=\"color: #000000;\">If your disk has crashed, and you need your music back, midi-carver may be your best bet.<\/span><\/p>\n<p><span style=\"color: #000000;\"><strong>Introduction<\/strong><\/span><\/p>\n<p><span style=\"color: #000000;\">Data recovery is tricky business.\u00a0 The best solution is not to have to do it at all: MAKE REGULAR BACKUPS.\u00a0 But if you have not done this &#8211; or even if you have and are unlucky enough\u00a0to have bad backups &#8211; then it becomes a much more painful process.<\/span><\/p>\n<p><span style=\"color: #000000;\">Starting with an image of the damaged source material (create using ddrescue or some other tool), the next best thing is to mount it locally.\u00a0\u00a0If that fails, one should try to\u00a0somehow rebuild \/ recover the FAT table, so the disk image may become usable again.\u00a0 If that fails,\u00a0the last resort is\u00a0&#8220;data carving&#8221;: searching the binary image for some indicators of file types, and then extracting them to disk.\u00a0 Some popular tools for doing this are foremost, scalpel, and PhotoRec.<\/span><\/p>\n<p><span style=\"color: #000000;\">midi-carver is a data carver specifically designed to recover MIDI files.\u00a0 It has a more intelligent extractor, in that it can skip damaged information and create\u00a0MIDI files\u00a0with the remaining good stuff.\u00a0 It can also, if you&#8217;re lucky, locate deleted items on the disk and recover them.<\/span><\/p>\n<p><span style=\"color: #000000;\"><strong>Features<\/strong><\/span><\/p>\n<ul>\n<li><span style=\"color: #000000;\">Attempts extraction of MIDI files from any binary blob<\/span><\/li>\n<li><span style=\"color: #000000;\">Locates &#8220;orphaned&#8221; MIDI tracks (where the header is missing) and creates a new MIDI header to contain the tracks<\/span><\/li>\n<li><span style=\"color: #000000;\">Copes gracefully with missing tracks by re-writing the header to the correct structure<\/span><\/li>\n<li><span style=\"color: #000000;\">Detects MIDIs which were overwritten by others, and truncates \/ splits into two separate tracks<\/span><\/li>\n<\/ul>\n<p><span style=\"color: #000000;\"><strong>Usage<\/strong><\/span><\/p>\n<p><span style=\"color: #000000;\">The hardest part of this would be compiling midi-carver.\u00a0 If you&#8217;re on Linux, FreeBSD or OSX this is easy.\u00a0 Download midi-carver.c, open a terminal and:<\/span><\/p>\n<p><span style=\"color: #000000;\">gcc -o midi-carver midi-carver.c<\/span><\/p>\n<p><span style=\"color: #000000;\">This will compile midi-carver into an application which can be run on a block file, like this:<\/span><\/p>\n<p><span style=\"color: #000000;\">.\/midi-carver &lt;filename&gt;<\/span><\/p>\n<p><span style=\"color: #000000;\">midi-carver will create a folder titled &#8220;mcut-out&#8221; in the same directory as &lt;filename&gt;, and save any MIDIs it finds into that folder.<\/span><\/p>\n<p><span style=\"color: #000000;\">Of course, you&#8217;ll need something to work on.\u00a0 I highly recommend using ddrescue on your damaged disk to create images.\u00a0 It&#8217;s very effective at getting the most undamaged data off the disk possible.\u00a0 Once you&#8217;ve done that, run midi-carver as described above, and look in the mcut-out to see if anything detected was usable.<\/span><\/p>\n<p><span style=\"color: #000000;\"><strong>Limitations<\/strong><\/span><\/p>\n<ul>\n<li><span style=\"color: #000000;\">midi-carver does not handle source fragmentation.\u00a0 This is ordinarily not a problem with iso images, but it is an issue with hard disks (and especially, floppy disk images) where files have been deleted and then added frequently.\u00a0 In this case the file system may break the files into blocks and rearrange them to effectively use all the disk capacity.\u00a0 midi-carver cannot read (or recreate) a file allocation table, so it cannot determine how to follow fragments.\u00a0 There is hope for a &#8220;last-block fragmented&#8221; recovery feature, but its usefulness would be slim.<\/span><\/li>\n<li><span style=\"color: #000000;\">midi-carver will happily terminate tracks without checking the actual MIDI events in the track.\u00a0 This produces a MIDI file which is structurally valid, but which media players may choke and die on, if the MIDI events in the track are not of the proper format.<\/span><\/li>\n<li><span style=\"color: #000000;\">There are a few pathological cases which may cause midi-carver to do the wrong thing &#8211; e.g. a tiny MIDI file embedded within a huge MIDI track will be missed.<\/span><\/li>\n<\/ul>\n<p><span style=\"color: #000000;\"><strong>Future Plans<\/strong><\/span><\/p>\n<p><span style=\"color: #000000;\">Fix limitations 2 and 3 above!\u00a0 This would mean adding a real\u00a0track validator that could follow MIDI events in a\u00a0track and then terminate it as soon as things appear to go sour.<\/span><\/p>\n<p><span style=\"color: #000000;\"><strong>Download<\/strong><\/span><\/p>\n<p><span style=\"color: #000000;\"><a href=\"http:\/\/greg-kennedy.com\/wordpress\/wp-content\/uploads\/2010\/12\/midi-carver.c\">midi-carver.c<\/a><\/span>\u00a0&#8211; 12.1 kb, version 1.0.<\/p>\n<p><span style=\"color: #000000;\"><strong>License<\/strong><\/span><\/p>\n<p><span style=\"color: #000000;\">It&#8217;s just 3-clause BSD.<\/span><\/p>\n<p>Copyright (c) 2010, Greg Kennedy<\/p>\n<p>All rights reserved.<\/p>\n<p>Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:<\/p>\n<ul>\n<li>Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.<\/li>\n<li>Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and\/or other materials provided with the distribution.<\/li>\n<li>Neither the name of midi-carver nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.<\/li>\n<\/ul>\n<p>THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS &#8220;AS IS&#8221; AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Overview MIDI-Carver is a tool for extracting (partial or complete) MIDI files from a binary block of data.\u00a0 It can be used against a solid data file &#8211; e.g. an ISO image &#8211; to attempt to retrieve MIDI data\u00a0into standalone files.\u00a0 It also can parse images of damaged filesystems and makes some attempt at recovery [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8],"tags":[],"class_list":["post-10","post","type-post","status-publish","format-standard","hentry","category-software"],"_links":{"self":[{"href":"https:\/\/greg-kennedy.com\/wordpress\/wp-json\/wp\/v2\/posts\/10","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/greg-kennedy.com\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/greg-kennedy.com\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/greg-kennedy.com\/wordpress\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/greg-kennedy.com\/wordpress\/wp-json\/wp\/v2\/comments?post=10"}],"version-history":[{"count":5,"href":"https:\/\/greg-kennedy.com\/wordpress\/wp-json\/wp\/v2\/posts\/10\/revisions"}],"predecessor-version":[{"id":80,"href":"https:\/\/greg-kennedy.com\/wordpress\/wp-json\/wp\/v2\/posts\/10\/revisions\/80"}],"wp:attachment":[{"href":"https:\/\/greg-kennedy.com\/wordpress\/wp-json\/wp\/v2\/media?parent=10"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/greg-kennedy.com\/wordpress\/wp-json\/wp\/v2\/categories?post=10"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/greg-kennedy.com\/wordpress\/wp-json\/wp\/v2\/tags?post=10"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}