october gamedev project reportback
i am very behind on gamedev project posts
okay the project from october 1st to october 15th was render buffers! it was a partial success.
going into it all i really knew was that rerendering was very expensive. i was guessing it was the part where my code entirely reallocated a buffer every time instead of just mutating values in the existing one, but i didn't really have any firm idea of how to do that in haskell.
specifically, because i wanted a big list of render actions, i was using existential quantification to hide the type signatures in render objects: each render object would store a shader with state
so anyway i solved that problem by just duplicating the value, after verifying that 'duplicating' the value was actually just taking a new pointer to the underlying buffer object and not, you know, actually copying the buffer itself. so then i could put one reference into the render object, and store the other in some non-quantified list that does have different constructors for each shader.
so that all works, but that's still not all the way to usability, because the actual goal is being able to update your buffers, which requires some additional infrastructure for tracking what's in the buffers and where and how to update them piecemeal. this part still doesn't work correctly; my landscape generation code seems to work differently when i'm rebuilding the entire landscape vs. when i'm only rebuilding a few tiles, even though it should be the exact same local context in the cases i'm using. also my indexing math isn't right or something so i end up corrupting all my vertex data somehow. but i did get far enough along to verify that yes mutating only a few indices is super fast compared to reallocating the entire buffer. that's not really a surprise in any way but it's nice to confirm that yes, that is where the hitching is from.
theoretically when i work on this next it'll be something like... trying to make values/a typeclass for piecemeal renderables, so that i can have a uniform interface for any kind of object that i can render only select parts of, which would let me very easily just be like "okay all these objects updated their data in some way, so regenerate only the relevant parts and rewrite only the relevant parts of their buffers" without having to actually think about buffer writes, which is my general goal with this code.
so, partial success. doesn't actually work, but the foundation is solid enough.
the project from october 16th to october 31st was data file syntax! it kind of petered out at the end as i got super into dwarf fortress again (more on that later) but i made a bunch of progress.
so as you might know, i've been messing around with procedural generation from data files for a while. it was my very first gamedev two-week project! huh, two years ago exactly.
i messed around with that concept a lot and rewrote the parser several times, but i never got it parsing precisely how i wanted (its whitespace rules were kind of finicky b/c newlines are sometimes terminators and sometimes not). also, as i added more features i started just bolting on syntax. "oh, i need some way to reference a generator's own attributes" "oh, i need some way to do list operations" "oh, i need some way to add or remove generators from the active item list". and i'd see that problem and add a new parsing rule for a new kind of action and keep going, but since i didn't actually plan out anything the parsing rules got super complex and the data files turned into this slapdash nightmare where there wasn't really any theme or metaphor to approach what it did; what it did was just "it does all the things i've specifically programmed it to do".
so i wanted to 1. get the syntax to be more legible even when it's doing something complex, and 2. to provide a sense that it's actually dealing with some kind of underlying conceptual object, rather than just running a bunch of rules i wrote in some order in some fashion.
this was kind of a weird one, since usually i wouldn't really consider "writing some fake data files and a parser to read them" to be like, work? it's pre-work. but in this case i've already written the internal combinatorics code that this would use, several times, and in fact the part where i was thinking about the data organization as not work and not important was what lead them to be such a tangled mess all the other times i worked on this concept.
this one i didn't really finish, because i got distracted, but i made a bunch of headway. i got the parser working and parsing in data in the ways i wanted, and i added only a limited amount of syntax. it's hard to say how much of a success this was without being able to actually run code, since a lot of the mess i made previously was due to the ad-hoc nature of realizing a needed a feature and bolting it on to the existing edifice, but now i at least have a better idea of how i'm gonna add in new syntax, rather than just going like "oh well i'll add
okay the project from october 1st to october 15th was render buffers! it was a partial success.
going into it all i really knew was that rerendering was very expensive. i was guessing it was the part where my code entirely reallocated a buffer every time instead of just mutating values in the existing one, but i didn't really have any firm idea of how to do that in haskell.
specifically, because i wanted a big list of render actions, i was using existential quantification to hide the type signatures in render objects: each render object would store a shader with state
s (like, texture references) and taking vertices of type a, as well as a static state value s and a render chunk representing a big buffer of type a values. and then it would make all those types vanish under quantification, so i could mix and match different shaders without having to do any special work with different constructors for each shader. and that was great for that purpose, but that also means i had thoroughly obliterated all the type information for my vertex buffers, so it was impossible to ever rewrite them. in fact, even if i had the information for the buffer i still couldn't rebuild the shader object, because the shader type a would no longer unify with the buffer type a! (the program has basically forgotten that they're the same type)so anyway i solved that problem by just duplicating the value, after verifying that 'duplicating' the value was actually just taking a new pointer to the underlying buffer object and not, you know, actually copying the buffer itself. so then i could put one reference into the render object, and store the other in some non-quantified list that does have different constructors for each shader.
so that all works, but that's still not all the way to usability, because the actual goal is being able to update your buffers, which requires some additional infrastructure for tracking what's in the buffers and where and how to update them piecemeal. this part still doesn't work correctly; my landscape generation code seems to work differently when i'm rebuilding the entire landscape vs. when i'm only rebuilding a few tiles, even though it should be the exact same local context in the cases i'm using. also my indexing math isn't right or something so i end up corrupting all my vertex data somehow. but i did get far enough along to verify that yes mutating only a few indices is super fast compared to reallocating the entire buffer. that's not really a surprise in any way but it's nice to confirm that yes, that is where the hitching is from.
theoretically when i work on this next it'll be something like... trying to make values/a typeclass for piecemeal renderables, so that i can have a uniform interface for any kind of object that i can render only select parts of, which would let me very easily just be like "okay all these objects updated their data in some way, so regenerate only the relevant parts and rewrite only the relevant parts of their buffers" without having to actually think about buffer writes, which is my general goal with this code.
so, partial success. doesn't actually work, but the foundation is solid enough.
the project from october 16th to october 31st was data file syntax! it kind of petered out at the end as i got super into dwarf fortress again (more on that later) but i made a bunch of progress.
so as you might know, i've been messing around with procedural generation from data files for a while. it was my very first gamedev two-week project! huh, two years ago exactly.
i messed around with that concept a lot and rewrote the parser several times, but i never got it parsing precisely how i wanted (its whitespace rules were kind of finicky b/c newlines are sometimes terminators and sometimes not). also, as i added more features i started just bolting on syntax. "oh, i need some way to reference a generator's own attributes" "oh, i need some way to do list operations" "oh, i need some way to add or remove generators from the active item list". and i'd see that problem and add a new parsing rule for a new kind of action and keep going, but since i didn't actually plan out anything the parsing rules got super complex and the data files turned into this slapdash nightmare where there wasn't really any theme or metaphor to approach what it did; what it did was just "it does all the things i've specifically programmed it to do".
so i wanted to 1. get the syntax to be more legible even when it's doing something complex, and 2. to provide a sense that it's actually dealing with some kind of underlying conceptual object, rather than just running a bunch of rules i wrote in some order in some fashion.
this was kind of a weird one, since usually i wouldn't really consider "writing some fake data files and a parser to read them" to be like, work? it's pre-work. but in this case i've already written the internal combinatorics code that this would use, several times, and in fact the part where i was thinking about the data organization as not work and not important was what lead them to be such a tangled mess all the other times i worked on this concept.
this one i didn't really finish, because i got distracted, but i made a bunch of headway. i got the parser working and parsing in data in the ways i wanted, and i added only a limited amount of syntax. it's hard to say how much of a success this was without being able to actually run code, since a lot of the mess i made previously was due to the ad-hoc nature of realizing a needed a feature and bolting it on to the existing edifice, but now i at least have a better idea of how i'm gonna add in new syntax, rather than just going like "oh well i'll add
$foo values" "and ^foo values also" "oh wait i also could use [foo] and {foo}". and so forth and so on.
