A call to arms

~ donluca

Link to previous part

We are (almost) set up to start archiving games again.
To quickly recap where we’re standing now:

  • We have a new accurate VGM logger in the blast’em emulator
  • We have an updated Deadfish hardware VGM Player which is able to completely disable the VDP (Video Processor) to minimize noise
  • Thanks to MDFourier, we now have the possibility of fine tuning the Deadfish VGM Player to achieve perfect timing
  • And we are working on a new workflow to make the recording process easier and faster

So, what’s missing?

To put it short: we need people to first check and then, where necessary, re-log VGMs.
Thankfully, some VGMs are already ok due to how the sound driver in some packs worked, but others, unfortunately, are off.
This means that at the beginning, we’ll be content just by compiling a list of games that are ok and others that need to be logged from scratch.

After that, if you thought that the process of recording and tagging files was long and boring, think again: VGM logging takes that boredom to an entirely new level.

I’ll briefly explain in very broad terms what’s needed to log a VGM, to give you an idea of the process:

  • Load the game’s ROM on blast’em
  • Hope to god that it has a sound test or a way to play back the music without sound effects
  • If it doesn’t, there’s a very big thread on Project2612 with all kind of hacks to let you play all the music in the game (even those hidden or unused!) without having to go through the whole game and without any SFX playing
  • You have to log the VGM and pay lots of attention to the music: the process of logging is very similar to the process of recording, but here you have to make sure that you record at least an entire loop (if the song is meant to loop)
  • Now, here comes the critical part: you have to trim the VGM you just logged so that it loops perfectly. We have a guide in the VGMrips Wiki about how this is done (link 1, link 2)
  • Finally, you have to add the GD3 tags (similar to the metatags you find in FLAC/MP3) to the tracks

Yes, this is incredibly time consuming and this is why we need help.

We have a Discord channel which is being graciously hosted on the Project2612 server and it’s pretty convenient because the server has lots of knowledgeable people how’ll be able to help you out, should you need it, to move your first steps into logging VGMs and hacking games to play the music back without SFX (although the guides I’ve linked above are pretty exhaustive).

To avoid getting the server destroyed by bots, the invite links will be shared through our Facebook and Twitter pages (or you can send me a Private Message in the forums).

If you were thinking about contributing, at this point I’d much rather have your efforts concentrated on the VGM part and I’ll be back to the recording process since it has caused much confusion among several contributors.
Plus, you won’t need any kind of recording equipment or buy anything in order to log VGMs: you just need a PC.
At this stage, even just checking the VGMs on Project2612 to see if they need to be re-logged would be really helpful.
Still, any kind of contribution will be more than welcome, as long as it follows our guidelines.

This is where we are and, again, this is absolutely too much for me alone to handle.

If you have any kind of question or doubt, please reach out via Facebook, Twitter, Forums or Discord and remember that while Discord might be great to handle instant communication, it’s pretty terrible for storing and finding information (which is why our Forums will still be left open).

Thanks everyone for getting through all those articles, I hope you’ve enjoyed them and I look forward getting back to archiving game music soon.

9 years in videogame music preservation – Part IV

~ donluca

Link to Part III

There’s nothing worse than trying to achieve a goal without the right tools. It makes the whole experience absolutely terrible and frustrating and the end result is seldom the best.

This is when Artemio (developer of the famous 240p Test Suite) contacted me on Twitter saying he had something he has been working on which could interest me.
The document he sent me titled MDFourier would be the beginning of a real revolution not just for our project, but for the entire world of videogame preservation, to the point were we wrote an entire article explaining what MDFourier is and how we could use it, featuring Artemio himself discussing the importance of the preservation of all forms of arts.

Thanks to MDFourier we could finally achieve what was previously unthinkable: perfect timing.
By now everyone knows we’ve been using VGMs which have their own shares of problems and limitations, but thanks to the valiant efforts of Project2612 and DeadFish Shitware, we were able to partially overcome them.
Or so we thought. Unfortunately MDFourier also uncovered a terrible truth: to put it short, almost all the VGMs in Project2612 which were ripped with KEGA (without getting too technical) had wrong timing.
We luckily managed to get around this (even without knowing it!) at the time because the DeadFish VGM Player had a feature to scale the playback speed to any desired amount, but this is truly devastating news because soon we’ll have to make a very difficult decision on whether to re-rip (most of) the entire Project2612 library or just let them be.
As I stated in the previous article, our project isn’t about just recording from real hardware anymore, it’s now about videogame music preservation and we need to do this in the most accurate way possible.

With MDFourier and the incredibly skilled community behind it, we were also able to make another huge discovery.
In the MDFourier article linked above, we discussed about what Mega Drive revision we should have used and we made some assumptions without any sort of proof.
Now, by sheer luck, we know that we’ve nailed it.
The fundamental specification about choosing a Mega Drive (which at the time of the article we completely missed) is that it should be able to play back correctly all the games available* and there are some games which won’t be played back correctly on Mega Drive VA3 and newer because SEGA decided to change the pre-amplification stage in VA3 making it a fair bit louder than the one in VA0-1-2.
This has lead to varying degrees of distortion on some tracks but, most glaringly, on Shadow of the Beast: it completely wrecked the entire game soundtrack, but on Mega Drive VA0-1-2 it plays absolutely fine without any distortion at all.
Another example of this is the Stage Clear track on Castlevania Bloodlines.

*Yes, I know, there are some games which were made specifically for the YM3438 and we’ll have to wrap our heads around this. We’ll need to make some very through research to decide what Mega Drive revision we should use for them. Most likely one of the newer Mega Drive Model 1s with the discrete YM3438 but don’t take my word on it.

So, you might ask, after all of this, where are we standing now?
In a pretty thrilling place!
We now have a new VGM logger, courtesy of blast’em, a well known cycle accurate emulator, which has been developed hand in hand with us and Project2612 and it’s absolutely perfect.
We won’t have to fear any kind of inaccuracies game-wise and VGM timing-wise anymore: we tested it thoroughly and it passed all the tests with flying marks.

So we’re all set up but there’s one very big issue: we’ll have to log all the VGMs again and, ideally, re-record all our releases from scratch.

This is something which is really disheartening but to live up to our goal, I feel like this is a necessary step. The biggest issue is that we can’t do this alone: it is just too much.

As of today, we’re still discussing with Project2612 the best course of action and what we’re going to do in the future, which is highly uncertain, as is the future of our project since it is tightly wired to Project2612.

Since this article is already pretty long as it is, there’s going to be a last part in this series which will focus on the future of our project and, most likely, a call to arms to our community (and others’) because we’re going to need all the help possible if we want to get through this.

Stay tuned as we’ll try to outline what we’re going to need and try to lay down some basic guidelines on how you can contribute to the future of our project.

9 years in videogame music preservation – Part III

~ donluca

Link to Part II

What happens when you go beyond what is humanly possible to hear, when no matter how hard you try, you can’t tell a difference?
Well, for most of us we’d say “that’s it, it’s good enough”.
And sure enough, there would always be cases when that “good” wouldn’t cut it.

After 7 years I started getting messages from emulator developers (both software and FPGA implementations) asking if they could use our work to fine tune their emulators. We were super stoked to know that we had reached a point where people would start using our releases for other purposes than just listening to the music.
Bear in mind that, until then, emulators never really took too much interest in shaping the audio output to try and get the sound as close to a real Mega Drive: they were happy enough with having the Yamaha YM2612 and PSG perfectly emulated, ignoring all the circuitry used in the audio output section of the console. This brought results which were pretty far from what a real Mega Drive sounded.

Up until last year we were still using very primitive tools (besides our ears – and our community’s) to understand if our work was accurate enough and it served us well so far.
The issue arose when we started to realize that the goal of the project was shifting once again.
All we ever wanted was to just record music from original hardware, trying to stay as close as possible to the original sound signature, recording using the most transparent equipment we could afford and let people enjoy our work.

When I started receiving those messages about our work being used in contexts where even a fraction of microsecond could make a difference I really wanted a more advanced tool to understand if we were doing things properly.
It became clear that our project wasn’t just about recording music as good as possible. We weren’t sharing music for our community’s pleasure.
We were doing something much more important: we were archiving Mega Drive music for preservation.

Although this seems a bit like an apocalyptic scenario, one day all our beloved old consoles might start failing to the point where they can’t be repaired anymore. Sure, emulators will still likely be around, but there wouldn’t be what we started referring to as a “reference”, a good, proper recording from original hardware.
A proper recording in an open source format such as FLAC will always be available to anyone and can be played back on any OS and architecture, even future ones which don’t exist today, because the source code of FLAC is publicly available and anyone can compile it and make a new player for a new OS or CPU.
And since our work has received lots of praise and seen use in several contexts, we just can’t screw this up anymore, we have to do things properly and with the right tools.

This is when everything changed again thanks to a new, incredible tool that suddenly became available, and the upcoming part will finally unveil the actual state of this project, what we’ve been doing so far and what’s going to happen in the future.

Hope you’ve enjoyed this small series of articles so far, stay tuned for the final part which is coming in the following days!

9 years in videogame music preservation – Part II

~ donluca

Link to Part I

How big of an impact can a 0.951% difference in speed have on the listener?

For a long time, we couldn’t really tell, that’s why we kept on using our modded PAL Mega Drive with region switch mod and swapped oscillator.
And yet… after a couple years of activity, people have started telling me they could hear that something was off in our releases and the best description they could give is that some instruments had wrong pitch.
They were, of course, right, because changing the speed at which a track plays means you end up changing the pitch as well.
At the 16-bit Audiohphile Project we could never tell this difference, but once more and more people started pointing this out, we just couldn’t ignore it anymore and we had to do some through investigation to understand what was wrong.

Mind you, those were times where something like MDFourier didn’t exist so we had to use our ears and listen through different tracks over and over again.
Eventually, it became clear that there was indeed a difference which, once you noticed it, you couldn’t just stop noticing it and you could hear it all over the work we’ve done.
At the time we were releasing games at full steam so we had a pretty substantial library of games which had to be ripped again.

This was really an eye opener for me: up until that moment, my goal was just to share Mega Drive soundtracks recorded from original hardware in the best way possible, but it became evident that what I considered “best” wasn’t good enough for our community and that’s when our project had its first big change in scope: it wasn’t anymore about myself sharing my recordings, it was about listening our community and improving over and over again to meet their (very!) high expectations.

If there was something which our community could hear which was wrong, it was time to step it up.
So I decided to do lots of research and eventually came up with using an early Japanese Mega Drive I , revision VA1 which needed a fix due to SEGA making an error in its output stage.
This wouldn’t have been possible without the help of Ace which wrote an exceptional thread in the Sega-16 forums about all the different revisions of Mega Drives and the fixes needed for each one.

And so our adventure continued and we had to introduce the concept of “Remasters” which were our old releases re-recorded from scratch with the new hardware.

You might think this would be the end of the story (and it could have been indeed) but our project managed to draw the attention of another group of people that I never thought they would use our material for their own work and this would eventually lead us into re-thinking everything we did up until then… again.

To be continued…

9 years in videogame music preservation – Part I

~ donluca

(Quick note: rather than making a single, huge post no one will read through, I have decided to make a mini-series of articles about the evolution of videogame music preservation and the role 16bap had in it. Several announcement will be made afterwards to keep everyone updated with the most important things going on in our project. Please stay tuned: huge changes are coming and we’re going to need help.)

On the 19th of October in 2012, the 16-bit Audiophile Project was born.
Its origins and goals were humble: in a world where soundtracks available on Youtube and forums were recorded mostly from (inaccurate) emulators, 16bap wanted to show how different real hardware sounded, and do it using the best recording equipment I could personally afford.

This wouldn’t have been possible without the huge efforts made by Project2612, who provided VGM logs of most of the Mega Drive / Genesis library, and Deadfish Shitware, who not only provided a software to play the VGM files on original hardware, but even modified it for us so that we could achieve more accurate timing.

And so, off we went, ripping and sharing music with the world recorded from authentic hardware.
I was astonished to find mixed responses to our work: there were those who were (and are) delighted with our releases and those that felt the sound was muffled, bass-heavy and lacked sparkle. It turned out that there are people that have never heard how a real Mega Drive sounded and their experience was limited to emulators such as Gens and KEGA which sounded way different from real hardware.
This was interesting to me, because while people each have different tastes and ideas on what kind of sound signature they enjoy, I understood that it was important to archive music in its original form, the way it played via its own circuitry.
I started wondering not only about recording quality, but other important aspects such as achieving perfect timing.
For casual listening purposes, no one would notice a 1% difference in speed (and, as a consequence, pitch), but when an expert ear starts listening to certain tracks critically, that 1% becomes easily detectable.

This became apparent after few years when some people started questioning our work and showing evidence that something was off.
Little we knew that there were people so passionate that went to the length of listening critically to our work and comparing it with their own Mega Drives. Their feedback would  lead to one of the turning point in our project, one which would eventually lead us into re-recording from scratch our releases in what became known as “Rematsters”…

To be continued…

A quick update on what’s been happening lately

~ donluca
Hello everyone and sorry for the long silence.
We’ve been hard at work with different groups on several things concerning our project (among many other aspects of videogame music preservation) and we’ve all been working towards something very big and important which will be announced in due time.
 
Meanwhile, I’ve added a disclaimer to our Release page: as we’ve been knowing for quite some time, all the “pre-remaster” releases have inaccuracies in their speed (and, as a consequence, their pitch) due to us using a PAL Mega Drive, despite the 60hz switch and crystal swap.
All the releases affected have been marked as such.
 
This was long due and we came to a point where we were forced to specify this because while even for the serious listener a 0.951% speed difference will probably go unnoticed, on the other hand our releases have been used for critical work (emulators, FPGA implementation, etc.) where even less than a 1% difference in speed has a huge impact.
 
We want to sincerely apologize to all the people who’ve relied on our releases and now have to double check their work to make sure that they were using either a Remaster or post-remaster release (all those after 3/11/2014 are very close to cycle accurate, with a discrepancy in speed of 0,00479% – newer ones will be cycle accurate).
We’ve been going through very big changes during those 8 years (yes, it’s been that long) and we’ve accumulated an incredibe amount of knowledge, but there’s always something new to learn and discover and now we’re going back to drawing board, looking at how we rip and record games from the Mega Drive, because we can’t afford to make any more mistakes and must make sure that our work is not only done through the best audio equipment possible, but is also done rigorously, making sure that all ends meet.
This means we’re re-hauling most of our own workflow and will make sure to be as transparent as possible as to what we’re doing and how we’re doing it so that people know exactly what they’re getting.
All releases are now on hold until we’ve untangled the mess and come up with something we know for sure that is completely accurate (or accurate enough that the differences are so small as to be negligible).
Thanks to all of you who’ve been following us and using our work and to all the fantastic people we’ve had the pleasure of working with so far.
We’ll be back with very substantial updates (hopefully) soon.