Today I finally finished up the ending related stuff I’ve been talking about for months. This was the last major task before I could ship the game. Except that I’ve just decided to add a bunch more tasks to the list.
It’s a bit of a long story, but one of my testers, Gregory Bednorz, has gone from writing a bit of guest music for the ending to being heavily involved in helping me improve and expand the rest of the music in the game. You can see a bit more about that and hear some samples of the new music (and the old!) in the latest video devlog:
I don’t want to rehash everything that I discussed in the above video, although I will say that I am still weighing different options as far as whether the game should release in some way before the new music is finished. There are reasons to release early and reasons to wait, and I am not 100% decided yet.
I was hoping to be a little less “all tech deep dives all day erry day”, but with that said, I think it could be enlightening to some to show how simple it was for me to implement a composer-editable configuration file format for the new music system.
I felt that the results would be better if Gregory was not only composing music tracks, but was also involved in the implementation of the tracks into the game. But since I didn’t want to send the whole Unity project back and forth or attempt to institute some type of version control system this late in development, I felt it would be good to have a configuration file system and dynamic asset importing.
The dynamic asset importing was actually more of a challenge than the parsing, so perhaps I will come back to that another time. Suffice to say that the development version of the game pulls in .wav files and generates a set of configurable audio layers based on the filenames. Then these filenames can be referenced and set up in a configuration file as follows:
zone Orchard1
{
osc_frequency_low 0.03
osc_frequency_high 0.07
osc_transition_interval 10
osc_transition_duration 3
add WindLow
add Orchard_FlutePadsLow
add Orchard_MatthewFluteSolo
tweak Orchard_MatthewFluteSolo offset_volume 0.5
tweak WindLow osc_low_volume 0.1
tweak WindLow osc_high_volume 0.4
}
layer WindLow
{
osc_low_volume 0.03
osc_high_volume 0.25
offset_volume 0
wind_edge_influence 0.5
}
layer Orchard_MatthewFluteSolo
{
osc_low_volume 0
osc_high_volume 0.1
offset_volume 0.4
wind_edge_influence 0.2
}
layer Orchard_FlutePadsLow
{
osc_low_volume 0.31
osc_high_volume 0.44
offset_volume 0
}
The way that this works, is that you set either a target zone or layer and then you configure settings for the target. Zones are areas in the world that contain only certain layers of the music and oscillate the volume of those layers up and down over time. Layers are just looping music tracks that have a volume that is tied to the high and low points of the current zone’s oscillator. The oscillator itself is just a sine wave which periodically blends into a new random frequency. This allows the music to change dynamically over time in a somewhat unpredictable way.
So, now let’s talk about parsing.
The main way that I kept parsing simple was that I decided that all state would be confined to individual lines, with only two exceptions: the current zone and the current layer. The brackets and indentation are purely for readability purposes and are ignored by the parser.
Because the inline code viewer on wordpress is horrible, and the pastebin one is slightly less horrible, here is a link to the entire relevant bit of code on pastebin.
The first thing that we do in terms of parsing is to point the C# standard StreamReader class at the file. I’m definitely not claiming this is the best or the most efficient way to parse a text file, but it’s what I’m familiar with from the save system, so I chose to reuse it here:
StreamReader sr = new StreamReader(file);
string line = "";
int active_line_index = 1;
while(sr.EndOfStream == false)
{
Once we get past this, we are now in the main loop, which is repeated for each line. First we create a cleaned version of the line by trimming any leading or trailing tabs and spaces.
stored_identifier = "";
line = sr.ReadLine();
line = line.TrimStart(' ', '\t'); //trim tabs and spaces
line = line.TrimEnd(' ', '\t');
int remainingChars = line.Length;
bool already_added_tweak = false;
Now we begin parsing through the line itself and breaking it down into tokens. A token is essentially just any single “word” in the file, so its just the next bit of text until there is a space. So we take the subset of the line until the next space and store that in a token string so we can interpret it later.
while(remainingChars > 0)
{
string token;
int tokenEnd = line.IndexOf(' ');
if(tokenEnd == -1)
{
token = line;
tokenEnd = line.Length;
}
else token = line.Remove(line.IndexOf(' '));
remainingChars -= token.Length;
if(tokenEnd+1 < line.Length)
{
line = line.Substring(tokenEnd+1);
remainingChars = line.Length;
}
else remainingChars = 0;
Now, we match the string against a set of keywords: #, zone, layer, add, tweak. The # is just a comment marker and is set to discard the rest of the line past it. The others all manipulate a line-level state variable which will be used in order to interpret the next token. If the token matches one of these keywords then we set the line-level state and parse out the next token on the line based on that state.
switch(token)
{
case "#":
remainingChars = 0;
break;
case "zone":
state = states.SET_ZONE;
break;
case "layer":
state = states.SET_LAYER;
break;
case "add":
state = states.ADD_LAYER;
break;
case "tweak":
state = states.TWEAK;
break;
default: //identifier, action depends on current state
When the token doesn’t match any of the keywords, this means it must be parsed differently depending on the current state. If we’ve already parsed a token on this line, and for example, that first token was zone, then the state will be SET_ZONE and we will parse the next token as though it was the identifier of a music zone that we want to set to be the target for future modification. This target, c_ActiveMusicZone, along with c_ActiveMusicLayer are the only state that is allowed to persist across multiple lines.
switch(state)
{
case states.SET_ZONE:
MusicZone targetZone = FindMusicZone(token);
bool created_new_music_zone = false;
if(targetZone == null)
{
targetZone = new MusicZone();
targetZone.name = token;
created_new_music_zone = true;
}
if(created_new_music_zone) musicZones.Add(targetZone);
else
{
targetZone.tweaks = new List<Tweak>();
targetZone.layers = new List<MusicLayer>();
}
c_ActiveMusicZone = targetZone;
break;
And we do the same if the first token was layer or add, setting the active music layer or adding a music layer to the active music zone, respectively.
case states.SET_LAYER:
MusicLayer targetLayer = FindMusicLayer(token);
if(targetLayer == null)
{
Debug.LogError("File: "+file+" Line: "+active_line_index+" Attempted to set a Music Layer active that does not exist! Please verify that the names match.");
}
else c_ActiveMusicLayer = targetLayer;
break;
case states.ADD_LAYER:
MusicLayer targetLayer2 = FindMusicLayer(token);
if(targetLayer2 == null)
{
Debug.LogError("File: "+file+" Line: "+active_line_index+" Attempted to add a music layer that does not exist! Please verify that the names match.");
}
if(c_ActiveMusicZone != null) c_ActiveMusicZone.layers.Add(targetLayer2);
break;
The next possible keyword and state would be tweak, which is a way for a zone to override the settings of a specific layer. This makes things a bit complex, because we have to parse out an identifier for tweaking as well as a value, which somewhat duplicates the default path when the token doesn’t match any keywords. I’ll confine the explanation of identifiers to that path, but here’s the tweak-related code anyhow:
case states.TWEAK:
if(already_added_tweak == false)
{
c_ActiveTweak = new Tweak();
c_ActiveTweak.target = FindMusicLayer(token);
if(c_ActiveTweak.target == null)
{
Debug.Log("File: "+file+" Line: "+active_line_index+" Unable to find target for tweaking \""+token+"\", perhaps it is misspelled?");
}
already_added_tweak = true;
}
else
{
if(c_ActiveTweak != null) //we only parse the rest of the line if we successfully found a tweak target
{
float value1;
if(float.TryParse(token, out value1))
{
c_ActiveTweak.value = value1;
switch(stored_identifier)
{
case "osc_high_volume":
c_ActiveTweak.type = tweakType.OSC_HIGH_VOLUME;
break;
case "osc_low_volume":
c_ActiveTweak.type = tweakType.OSC_LOW_VOLUME;;
break;
case "offset_volume":
c_ActiveTweak.type = tweakType.OFFSET_VOLUME;
break;
case "blend_duration":
c_ActiveTweak.type = tweakType.BLEND_DURATION;
break;
case "wind_edge_influence":
c_ActiveTweak.type = tweakType.WIND_EDGE_INFLUENCE;
break;
}
if(c_ActiveMusicZone != null) c_ActiveMusicZone.tweaks.Add(c_ActiveTweak);
}
else stored_identifier = token;
}
}
break;
Next we have the general case, which is when we didn’t match a keyword like layer, tweak, or add. In this case, it’s assumed that the token is an identifier for a variable or the value that we want to set a variable to. So first we attempt to parse the token as a value, and if that fails then we store the token as the identifier of a variable which we will set after we parse the next token on the line. There is no overlap between layers and zones when it comes to the names of variables, so as long as there is a current active zone or active layer, we can set the appropriate values just based on the identifier.
default:
//we must assume that the token is intended to identify a variable or a value, so we will check against a list of known variable names
float value;
if(float.TryParse(token, out value))
{
switch(stored_identifier)
{
case "osc_frequency_low":
if (c_ActiveMusicZone != null) c_ActiveMusicZone.osc_frequency_low = value;
break;
case "osc_frequency_high":
if (c_ActiveMusicZone != null) c_ActiveMusicZone.osc_frequency_high = value;
break;
case "osc_transition_interval":
if (c_ActiveMusicZone != null) c_ActiveMusicZone.osc_transition_interval = value;
break;
case "osc_transition_duration":
if (c_ActiveMusicZone != null) c_ActiveMusicZone.osc_transition_duration = value;
break;
case "osc_low_volume":
if(c_ActiveMusicLayer != null) c_ActiveMusicLayer.osc_low_volume = value;
break;
case "osc_high_volume":
if(c_ActiveMusicLayer != null) c_ActiveMusicLayer.osc_high_volume = value;
break;
case "offset_volume":
if(c_ActiveMusicLayer != null) c_ActiveMusicLayer.offset_volume = value;
break;
case "blend_duration":
if(c_ActiveMusicLayer != null) c_ActiveMusicLayer.blend_duration = value;
break;
case "wind_edge_influence":
if(c_ActiveMusicLayer != null) c_ActiveMusicLayer.wind_edge_influence = value;
break;
default:
Debug.LogError("File: "+file+" Line: "+active_line_index+" Attempting to modify an unknown variable, please verify the name!");
break;
}
}
else stored_identifier = token;
break;
And that pretty much covers the parser, apart from that we reset the state between lines and increment the line counter so we can report errors a bit more helpfully.
Is it even remotely bulletproof? No. But for something quick and dirty that was mostly written in a day, I think it gets the job done, and hopefully can serve as somewhat of an inspiration against the idea that one should always “just use JSON” or that “parsing is hard”, or whatever other nonsense excuse you might have to not just write your own format.
If I were to think of some things that definitely could be improved, the main would be making sure that all error cases are handled better. For the most part, the format is simple enough that even if there is a major error, the file should still parse for the most part and just report the error without putting the game into a broken state. However, there may be some error states which I am not reporting, which could bite us in the butt on down the line.
Another avenue for improvement would be to have a file version marker. Since I expected these files to be human edited, I didn’t want to mark up all of the variables with some type of version information, but at least a file level version marker could be useful if there are some changes to the format that I want to make. This could facilitate automatic conversions of older files to a new format. With that said, this is a relatively minor concern, since there’s really only Gregory and myself working with these files, and I can easily just convert the files and pass them back to Gregory.
Taiji 