Table of Contents
The file I/O library in X-Forge is called XFcFile. It has been modeled after the standard C library fopen()/fclose() style file handling. No C++ file stream extensions are included.
In addition to file I/O, the library makes sure all the data is always found in the same directory as where the binary executable lies on all of the platforms, and also transparently wraps compressed file library I/O as well. The library also handles any file system inconsistencies (such as slash vs backslash for directory separation).
For portability, applications should always use slash separator for directories and keep all filenames case sensitive. All of the supported platforms can handle long filenames, but in order to keep possible porting problems minimal, it is recommended that filenames are kept under 32 characters long, and they should only include alphabets (a-z only), numbers and underscores, followed by a point and a three letter extension.
The files are opened, and XFcFile object is created, using the XFcFile class static member open(). The syntax is as follows:
XFcFile *f = XFcFile::open(XFCSTR("filename.dat"),XFCSTR("rb"));
Since the two parameters are of CHAR type, the static data strings ("filename.dat" and "rb") are enclosed in XFCSTR() macro, which will take care of ASCII-unicode conversion should one be needed.
Currently all of the platforms supported by X-Forge are internally ASCII. The decision whether we should change all of the platforms to unicode to ease any possible localization effort is still pending. For the time being it is safe to remove the XFCSTR() macros, but if the change to unicode is made, all of this code will break. The compiler will alert if this happens, though.
The filename can include directories, but the 'root' directory is still the directory where the executable is. Internally, the core adds the current directory to the filename before passing it to the operating system. Thus, filename.dat turns into /my documents/filename.dat. Obviously, if you try to open ../filename.dat it will turn into /my documents/../filename.dat, which most probably won't work. If you absolutely need to access files outside the application's directory, you can bypass the filename mangling completely by starting the filename with two slashes (for example, //c:\config.sys). This code will not be portable across platforms, however.
The files can exist in either on the disk as separate files (in the same directory as the executable, or its subdirectories), or inside compressed file library (CFL) files. The system first checks files on disk, and then inside CFLs, in order to make development easier. In order for the system to access CFL files they must first be mounted using XFcCore::openCFL() call.
If you want to force the system to open from disk or from a CFL, you can use XFcFile::openDiskFile() or XFcFile::openCFLFile() calls instead of the normal open().
Currently, when you open a file that is inside a CFL, the whole file will be uncompressed at that time into memory. This is necessary in order to support random access with the way that CFL is currently implemented. Keeping many files open and/or using large files can eat a lot of runtime memory. If you need to stream a lot of data, it is best not to use CFL for that.
File access modes work as follows:
"r" - Open for reading. If file does not exist, opening fails. "w" - Open for writing. If file existed, its contents are destroyed. "a" - Open for appending. Opens file and sets the write head at the end of the file. If the file does not exist, it is created. "r+" - Opens file for reading and writing. File must exist before the call. "w+" - Same as r+, except that if file existed its contents are destroyed. "a+" - Same as r+, except that it creates the file if it doesn't exist, and write head is positioned at the end of the file. "b" - Consider the data to be binary, and don't do any character conversion.
Files that are inside a CFL file are read-only, so if the file access flag contains w, a or +, only files on disk are accessed.
Due to the inconsistency of ASCII-mode file I/O, it is highly recommended that all files are always opened using binary mode.
Files inside CFLs never do any ASCII-mode conversions.
Files can be closed, and XFcFile object freed, by using the XFcFile class function close().
f->close();
Any memory allocated by opening the file is also freed.
File seeking is done via the XFcFile class function seek().
f->seek(offset, seek type);
The offset is the amount of bytes to move in either direction.
The following seek types are supported:
SEEK_CUR - Current position in file SEEK_END - End of file SEEK_SET - Start of file
In order to find out where the file pointer is at the moment, you can use the tell() call.
INT32 position = f->tell();
Data can be read either byte by byte using getChar() call, or in a chunk using read() call. The getChar() function represents the fgetc() call on stdlib, but due to the fact that getc() is implemented as a macro on some platforms we cannot use the same name.
Syntax for getChar() is as follows:
INT32 data = f->getChar();
Please note that getChar() does not return CHAR type data (which could be 8 or 16 bit depending on whether the platform is unicode), but 8-bit integers. Another return value can be EOF, which is outside the 8-bit range (it is defined by stdio.h as (-1)).
Syntax for read() is as follows:
INT32 count = f->read(targetBuffer, blockSize, blockCount);
The returned value is the number of blocks read. Usually programs ignore this information, using file size information instead of the return value to figure out how much data is available. The number of bytes read is blockSize * blockCount.
Data can be written using the putChar() or write() functions. The syntax for putChar() is as follows:
INT32 result = f->putChar(value);
The return value will be EOF if there was an error, or the same value as written if not.
The syntax for write() is as follows:
INT32 count = f->write(buffer, blockSize, blockCount);
The returned value is the number of blocks written. If blockCount is not equal to count, the system ran out of disk space.
Files can be renamed using the XFcFile class static function rename().
INT result = XFcFile::rename(XFCSTR("old.dat"), XFCSTR("new.dat"));
The result value is 0 if successful, non-zero otherwise.
Trying to rename files that are inside a CFL will fail.
Removing or renaming a CFL file that has been mounted may cause undefined behavior.
Files can be deleted using the XFcFile class static function remove().
INT result = XFcFile::remove(XFCSTR("unneeded.dat"));
The result value is 0 if successful, non-zero otherwise.
While remove() can be used to destroy directories in some platforms, this functionality may not work on others.
Trying to delete files that are inside a CFL will fail.
Removing or renaming a CFL file that has been mounted may cause undefined behavior.
You can check whether a file exists using the XFcFile static function fileExists().
INT result = XFcFile::fileExists(XFCSTR("myfile.dat"));
The result is 0 if file does not exist, 1 if it can be found inside a CFL, and 2 if the file can be found on disk. If file can be found in both places, the function will return 2, since files are first accessed from disk.
File finding can be done using the XFcFileFind class. The class can be used to enumerate through CFL and disk file directories.
The syntax is as follows:
XFcFileFind *ff = XFcFileFind::create(filemask, flags); CHAR *filename = ff->next(); while (filename != NULL) { // do something with filename filename = ff->next(); } delete ff;
File mask supports DOS-style file masking, ie. you can use * and ? wildcards. The ? wildcard matches one character while the * wildcard matches any number of characters. "fo?.dat" matches "foo.dat" but not "fooo.dat"; "fo*.dat" matches both "fo.dat" and "fooooooooo.dat". Unlike DOS, you can use several * wildcard characters, so "f*o*o*.dat" matches correctly to "foobar.dat" and ignores "figaro.dat" correctly.
Different flags can be used to alter the XFcFileFind's behavior.
XFCFF_DISK - find files on disk directory XFCFF_CFL - find files inside CFL directory XFCFF_REMOVEDUPLICATES - remove duplicates (if the same file exists on both disk and CFL) XFCFF_IGNORECASE - perform case-insensitive file name matching
Some platforms iterate through disk directories and others don't.
You can get the size of a file using the XFcFile class static function getFileSize().
INT32 size = XFcFile::getFileSize(XFCSTR("myfile.dat"));
The size is 0 for files that don't exist. If file of the same name exists both inside CFL and on disk, the size of the disk file is returned.
The XFcFile class includes a bunch of data serialization extensions which are meant to be used in any data file use. Any platform-specific data conversion is handled by these functions as well.
Please note that the REAL data type is never read or written directly, as its internal specification may change. Such data should always be stored as FLOAT32.
INT32 writeFLOAT32(FLOAT32 aValue); INT32 writeFLOAT64(FLOAT64 aValue); INT32 writeINT32(INT32 aValue); INT32 writeINT16(INT16 aValue); INT32 writeINT8(INT8 aValue); INT32 writeUINT32(UINT32 aValue); INT32 writeUINT16(UINT16 aValue); INT32 writeUINT8(UINT8 aValue);
Same can be said about reading functions. The REAL type always handles conversion itself, so you can use readFLOAT32() and writeFLOAT32() directly with them.
FLOAT32 readFLOAT32(); FLOAT64 readFLOAT64(); INT32 readINT32(); INT16 readINT16(); INT8 readINT8(); UINT32 readUINT32(); UINT16 readUINT16(); UINT8 readUINT8();
The read functions also come in read-to-reference flavor which can be used to mirror the write functions directly. The returned value is the bytes read.
INT32 readFLOAT32(FLOAT32 &aValue); INT32 readFLOAT64(FLOAT64 &aValue); INT32 readINT32(INT32 &aValue); INT32 readINT16(INT16 &aValue); INT32 readINT8(INT8 &aValue); INT32 readUINT32(UINT32 &aValue); INT32 readUINT16(UINT16 &aValue); INT32 readUINT8(UINT8 &aValue);
For string reading and writing, the following functions handle reading and writing of zero-terminated strings. The zero is also written so that the string can be read as well. CHAR-strings are always stored as CHAR16 and the read/write functions handle the conversion to whatever format is used internally. To store the string as 8-bit characters, use CHAR8 functions instead.
void writeCHARString(CHAR *aValue); void writeCHAR8String(CHAR8 *aValue); void writeCHAR16String(CHAR16 *aValue); CHAR * readCHARString(); CHAR8 * readCHAR8String(); CHAR16 * readCHAR16String();
Compressed file libraries are meant as frozen, compressed storage for game resources. Once created, they are frozen and cannot be updated.
CFLs do not support streaming, and thus the only way to get data from a CFL is to read a whole file at once. XFcFile wraps around this functionality by storing the file inside a RAM buffer and taking care about file pointers in this ram buffer.
CFL can be extended by writing new filters. By default, the only filters supported are 'No compression' and 'ZLib compression'. In addition to compression, CFL supports preprocess and encryption filters. Preprocess filters can be used to transform data into more compressable form, and encryption filters can be used to make data files more secure.
Please note that no amount of encryption will make the data files un-rippable if your program uses the files. You can, however, make the data files uneditable by using some public key encryption algorithm. If you plan to do so, make sure you open the files from CFL and not from disk.
CFL applies the filters also to the directory data, reducing size and optionally enhancing security.
You can also store data files inside a CFL file in uncompressed form, and request XFcFile pointer to it, or you can ask for the offset inside the .CFL file and do the seeking yourself.
The core includes, internally, one instance of XFcCFL object, which is linked to the XFcFile system. You can open CFL files to this object by using the XFcCore::openCFL() calls. If you want to, you can create new instances of XFcCFL objects, but accessing these files is not as handy as via the XFcFile system.
Some plans on changing the way CFLs work exist, but their implementation is still open.
You can load a file at once using the XFcFile class static function getFile().
INT8 *data = XFcFile::getFile(XFCSTR("myfile.dat"));
This will work on both files inside a CFL and also for files on disk. This function is typically used with XFcFile class function getFileSize().
If you use your own CFL object, you can get the file using the CFL object's getFile() function:
INT8 *data = myCFLObject->getFile(XFCSTR("myfile.dat"));
Using the CFL object you can also load the file into a pre-allocated buffer.
INT32 writtenBytes = getFile(XFCSTR("myfile.dat"), buffer, maxSize);
If you wish to include some data with your application that is infeasible to compress with CFL (due to the memory buffer problem or otherwise), but still want to keep distribution file count to a minimum, you can store the file inside CFL in uncompressed form and then access it directly by opening the .CFL file and seeking to the file.
XFcCFL object supports this with getFileOfs() function.
INT32 offset = myCFLObject->getFileOfs(XFCSTR("myfile.dat")); XFcFile *f = XFcFile::open(XFCSTR("mycfl.cfl"), XFCSTR("rb")); f->seek(offset, SEEK_SET) ..
The above clip can be simplified by using the getFilePtr() call:
XFcFile *f = myCFLObject->getFilePtr(XFCSTR("myfile.dat"));
If you get a file pointer using getFilePtr() you must close it using the dropFilePtr() call:
myCFLObject->dropFilePtr(f);
Closing the file pointer directly may result in unpredictable behavior.
You can also create CFL files at runtime. This can be useful when making saved games. If your game objects require several kilobytes of data, it's best to have them compressed.
CFLs are created using the XFcCFLMaker class. The creation process follows the following pattern:
XFcCFLMaker *maker; maker = CFLMaker::create(XFCSTR("mysavefile.cfl")); maker->store(XFCSTR("thisdata"), thisdata, thisdatasize, compressFlags); maker->store(XFCSTR("thatdata"), thatdata, thatdatasize, compressFlags); maker->store(XFCSTR("somedata"), somedata, somedatasize, compressFlags); maker->finish(compressFlags);
The create call creates the file and initializes the header; store calls compress and store the data and build the directory in memory. Finally, the finish call serializes the directory, compresses it, stores it, updates header, closes the file and destroys the XFcCFLMaker object.