data storage

[mattheweston]

I'm trying to figure out what to use for storing the data in my game. It's going to be a football game and I need to store all the stats as well as rosters and league information. I am currently debating between the following three options.

1. Roll my own
2. Access DB
3. SQLite

The game is going to have to create files that are able to be copied and ran on multiple computers like saving game data to a memory card and then using that on someone else's console.

Anyone ever tried/had to use a db for their game? Thoughts?

[superLED]

What are you programming in?

[mattheweston]

C#...I'm leaning more so toward using SQLite.

[Falco Girgis]

...Roll your own.

[LeonBlade]

...Roll your own.

This. There's no need for a database, you're better off making something yourself which does exactly what you need without all the unneeded SQL stuff which is slow and useless for a game.

[lotios611]

...Roll your own.

This. There's no need for a database, you're better off making something yourself which does exactly what you need without all the unneeded SQL stuff which is slow and useless for a game.

When should you use a database then?

[Aleios]

...Roll your own.

This. There's no need for a database, you're better off making something yourself which does exactly what you need without all the unneeded SQL stuff which is slow and useless for a game.

When should you use a database then?

When in need of storing massive amounts of data. MMO's typically use databases, and all the crap for that is handled server side.

In this case he could simply make an XML file, or even just a text file. Chuck the data in some format in either a text file or XML file and it can be copied to many machines with ease.

[dandymcgee]

...Roll your own.

This. There's no need for a database, you're better off making something yourself which does exactly what you need without all the unneeded SQL stuff which is slow and useless for a game.

When should you use a database then?

When in need of storing massive amounts of data. MMO's typically use databases, and all the crap for that is handled server side.

In this case he could simply make an XML file, or even just a text file. Chuck the data in some format in either a text file or XML file and it can be copied to many machines with ease.

That's just one of a million different places a database is used. Databases have a ton of advantages over a simple XML file. I would recommend using a database for any project require a large, indexed data store. If you need to search through thousands of records for a particular one, XML will perform like shit compared to SQLite. If you just need to store settings for one user, a text database (XML, BIN, TXT, whatever format it may be) will be more suited to your project.

@mathewweston In your case it depends one what you're doing with the league / roster information. I'd assume you're just loading it up for a particular "level" in the game then using it from there. In that case XML would work just fine.

[Falco Girgis]

Unless you are making an MMO or other sort of game where data is stored on a server, there is absolutely no need to use something as large and slow as a SQL database in an indie game.

And I also think XML is over-used. It's expensive and slow as hell to parse. It's overkill in almost every scenario that I've seen it used... If you are using a Level Editor or Toolkit, why in god's name would you go with ANYTHING other than a straight binary file? It's WAY smaller and WAY faster. Without debate, the optimal solution. The entire point of a Level Editor is to prevent you from having to create assets by hand... which is the only reason you would ever use something like XML.

[szdarkhack]

And I also think XML is over-used. It's expensive and slow as hell to parse. It's overkill in almost every scenario that I've seen it used... If you are using a Level Editor or Toolkit, why in god's name would you go with ANYTHING other than a straight binary file? It's WAY smaller and WAY faster. Without debate, the optimal solution. The entire point of a Level Editor is to prevent you from having to create assets by hand... which is the only reason you would ever use something like XML.

Because it compresses well

Sorry, i just had to say it On a serious note though, it depends on the situation. If you need to store a lot of data and only read them just a few times, xml works fine (for example, a configuration file). But for stuff that you need to load quickly (like tilemaps) you're definitely better off with a binary file. It will save you a lot of loading time.

[LeonBlade]

Unless you are making an MMO or other sort of game where data is stored on a server, there is absolutely no need to use something as large and slow as a SQL database in an indie game.

And I also think XML is over-used. It's expensive and slow as hell to parse. It's overkill in almost every scenario that I've seen it used... If you are using a Level Editor or Toolkit, why in god's name would you go with ANYTHING other than a straight binary file? It's WAY smaller and WAY faster. Without debate, the optimal solution. The entire point of a Level Editor is to prevent you from having to create assets by hand... which is the only reason you would ever use something like XML.

I agree with Falco, really the only time you should be using a database is if the information is going to be pulled across the Internet aka an MMO. In your case it makes the most sense to just pack together a binary of what you need and then parse it out. You save space, you save time by not using SQL and having to construct queries to pull out small bits of data, and it fits your needs and nothing more or less.

I also agree about XML being overused. XML is really only useful when it comes to things that you'll be changing manually. XML is a markup language, it's useful for changing manually because it's easy to read. In your case, the only thing that should be modifying your files is your application/game and nothing else really.

I hate seeing people talk about how they made an XML file for their level editor or for whatever they needed. And then they have to go and parse through the XML elements to grab information either manually with matching the XML string and not actually using proper XML parsing, or getting an XML parser that can spit out an array that you can pull data from. All of that is really slow and pointless, honestly a huge waste of time.

If you want to prototype something, or play around, at least make a comma delimited text file or something, not full blown XML. Then later once you realize what you need in your files, you can create a file that you can read in and you wont have to worry about being shit deep in XML for all your files and now it comes time to make everything into a binary.

[Falco Girgis]

And I also think XML is over-used. It's expensive and slow as hell to parse. It's overkill in almost every scenario that I've seen it used... If you are using a Level Editor or Toolkit, why in god's name would you go with ANYTHING other than a straight binary file? It's WAY smaller and WAY faster. Without debate, the optimal solution. The entire point of a Level Editor is to prevent you from having to create assets by hand... which is the only reason you would ever use something like XML.

Because it compresses well

Sorry, i just had to say it On a serious note though, it depends on the situation. If you need to store a lot of data and only read them just a few times, xml works fine (for example, a configuration file). But for stuff that you need to load quickly (like tilemaps) you're definitely better off with a binary file. It will save you a lot of loading time.

XML compresses better than a custom, binary-based file packed to 100% efficiency with not a single wasted character? And what about having to decompress this? And who says you can't compress your already compressed binary file?

And sure, XML works "fine" when you're storing a lot of data and only reading them a few times like a config file... but goddamn, including an entire XML parsing library to load one file seems like overkill to me. Call me old-fashioned and low level, but that's definitely not something I will agree with. XML definitely has its place in software development... but unless you don't have a Level Editor, I don't see it in game development...

I hate seeing people talk about how they made an XML file for their level editor or for whatever they needed. And then they have to go and parse through the XML elements to grab information either manually with matching the XML string and not actually using proper XML parsing, or getting an XML parser that can spit out an array that you can pull data from. All of that is really slow and pointless, honestly a huge waste of time.

THANK YOU. I think this is the pathological worst-case. What is the point of a Level Editor? To automatically generate assets for your game/engine. Why the hell would you ever mess with these files manually? You are gaining absolutely nothing good from using XML in this scenario. It requires an additional runtime dependency in both your engine and editor, slows down your loadtimes, and bloats the hell out of your files. You've added NOTHING useful.

With a Level Editor, you have a prime opportunity to compress the hell out of your files and make everything as efficient and tightly-packed as possible.

[szdarkhack]

I was kidding about the xml compressability... It's just what "fans" say all the time and i'm sick of hearing it. Using it for configuration purposes is in my opinion perfectly reasonable if you want them to be human-readable, given that the amount/structure of data justifies it. For example, if you can do it with a name-value pair file then do it, it'll be faster. There are cases, however, when you might need to have a more complex, hierarchical structure in your stored data, in which case XML is a pretty fair choice. In general though, the simpler you can afford to make things, the faster/smaller they will be.

Again, i'm in no way implying or suggesting that people should use xml for game data, you're clearly better off with binary files there. I thought i made that clear, but here it is once more, just to make sure.

[LeonBlade]

I was kidding about the xml compressability... It's just what "fans" say all the time and i'm sick of hearing it. Using it for configuration purposes is in my opinion perfectly reasonable if you want them to be human-readable, given that the amount/structure of data justifies it. For example, if you can do it with a name-value pair file then do it, it'll be faster. There are cases, however, when you might need to have a more complex, hierarchical structure in your stored data, in which case XML is a pretty fair choice. In general though, the simpler you can afford to make things, the faster/smaller they will be.

Again, i'm in no way implying or suggesting that people should use xml for game data, you're clearly better off with binary files there. I thought i made that clear, but here it is once more, just to make sure.

Yeah I knew you were joking.

[Falco Girgis]

I was kidding about the xml compressability... It's just what "fans" say all the time and i'm sick of hearing it. Using it for configuration purposes is in my opinion perfectly reasonable if you want them to be human-readable, given that the amount/structure of data justifies it. For example, if you can do it with a name-value pair file then do it, it'll be faster. There are cases, however, when you might need to have a more complex, hierarchical structure in your stored data, in which case XML is a pretty fair choice. In general though, the simpler you can afford to make things, the faster/smaller they will be.

Again, i'm in no way implying or suggesting that people should use xml for game data, you're clearly better off with binary files there. I thought i made that clear, but here it is once more, just to make sure.

Yeah I knew you were joking.

I didn't.

Myyy bad.