Archive for the ‘C++’ Category

Extending std::ostream HOWTO

2010/07/01 1 comment

Extending std::ostream – HOWTO

iostream class design

First things to know.

  1. All basic_* classes are actually the template version of the usual classes. So it’s simpler to work with specialized classes for early development.
    That is instead of working with the class std::basic_ostream<<class _CharT, class _Traits> works with std::ostream.
    It’s actually just because of typedef basic_ostream<char, char_traits<char> >  ostream;
  2. Streams are there to hide actual implementation of communicating with the outside world, and separate communication from the logic of your application. These type of communication requires buffer for decent performance, and these are implemented in streambuf’s children classes. Decent STL implementations should provide you with stringbuf and filebuf as a minimum.

Therefore we could represent dataflows like this :

App ==>> ostream -->> ostreambuf --- ES
ES --- istreambuf -->> istream ==>> App

ES stands here for external systems, whether it is the OS, or any other
system you need to connect to without bothering about connection intricacies

Design choice

For simplicity sake, we will work while considering a very simple usecase :

MyOStreamBuf sb;
MyOstream mos(&sb);
int value = 42;
mos << "value :" << value << std::endl;

The simplest way to extend a stream usuallty is to extend streambuf with
“mystreambuf” and provide a stream extension “mystream” to work on it the same
way that ostream works on ostreambuf. This way the user of mystream will use it
as he could use std::ostream, and the implementation details will be hidden.

Streambuf Extension : What is needed ?

There is actually very little required to extend std::ostreambuf. Here is an
exemple with a nullstream implementation (template version), which does

streambuf::overflow is called whenever the buffer is full and action needs to
be taken to enable more input. Depending on the kind of buffer we are
manipulating, this can mean flushing, extending buffer memory, etc.
It is expected that any class implementing streambuf will overload
streambuf::overflow(typename trais::int_type c)

Other usually overloaded functions, when writing streambuf implementations are streambuf protected virtual function. it is advised not to overload any public function, as they guaranty the proper behaviour of std::streambuf when used by a std::stream. In their specialized version :

  1. virtual std::streamsize xsputn ( const char * s, std::streamsize n ); is called whenever a character needs to be written into the std::streambuf via std::streambuf::sputn(), called from ostream& operator<<(ostream& , Type inst). Special formatting can be done here.
    The return status is expected to be n. If it is 0 the stream using that buffer will set ios::failbit to true, and prevent any more use of that stream.
  2. virtual int sync (); is called whenever the buffer needs to be syncrhonised with the ES, usually via std::ostreambuf::pubsync() called from std::ostream::flush() (called by manipulator std::endl for exemple)

On the use case :

(mos << "value :" ) // calls xsputn(), ( or overflow() if buffer is full)
<< value            //calls xsputn(),
//note the formatting from int to char* has been made by operator<<
<< std::endl;       //calls overflow() and then flush()

Stream Extension : Wrapping My streambuf extension

Here is an exemple with a nullstream implementation. Watch how the constructor needs to call init() on the buffer type, to bind the stream to it.

  1. An explicit constructor of stream can also be created and take as parameter the proper streambuf type. In the specialized version :
    explicit Mystream(Mystreambuf* lsb);
  2. It is good to overload also Mystreambuf* rdbuf() const to return the actual Mystreambuf type instead of the original std::streambuf
  3. For the same reason, you should also overload :
    //Put character (public member function)
    Mystream& put ( char c );
    //Write block of data (public member function)
    Mystream& write ( const char* s , std::streamsize n );
    //Flush output stream buffer (public member function)
    Mystream& flush ( );


You should now be able to write your own streambuf implementation, and a stream to allow the use of operator<< for all basic types, as well as all the standard manipulators defined in STL. And this without coding any operator<< function or manipulator.

std::ostreambuf is extendable quite simply. It is possible but more complicated to extend std::ostream, as it would probably require rewriting a lot of code. Therefore if an extended feature can be put into streambuf, it seems much simpler to do so.

This has been used in the AML project to write logstreambuf and logstream. At the time of writing the development of the Logging feature is still going on…

Categories: C++

Busy C++ coding and testing…

2010/06/20 1 comment

I have been pretty busy lately, mostly with AML. Now that most features are there, and that a stable structure is emerging, the main goal is to write a full test suite, and improve code quality and documentation. This way we should be able to spot all differences introduced by new code very quickly. And also the project being multiplatform, it will be tremendously usefull to spot differences between different platforms easily.

But a clever complete test suite for a “driver-output based software” (understand : a software which main job is to output things on hardware through driver) is very difficult to make. To be able to automate such a test suite, you also need to get back the information you just outputted to check it. I started doing that for images, but I don’t have much idea yet about how I can implement tests about keyboard input, joypad, or font rendering for exemple… audio anyone ?

And this is actually much more work than I anticipated. But it will save even more later down the road…
We decided not to use cppunit just yet, because it would be one more dependency and one more library to learn. Better to focus on the tests themselves, we can always start using cppunit later.

I recently have been working on a logger for the Core part of AML. The main features were :

  • use of log level filtering of message
  • use of stream operator seamlessly
  • insertion of custom prefix
  • insertion of date, threadID (later), and other useful information to identify future issues
  • default output to clog (console)
  • optional output to file
  • being able to extend the log system to use new outputs (syslog, win32dbg, etc.)

So I decided to extend the ostream class from STL. But to do this without having to overload each operator<< it was much better to extend streambuf as well. However I ended up to extend stringbuf directly, as I wanted an stringstream-like input buffer,  and on sync() copy everything to the streambuf chosen as sink. Not an easy work at first, because it can depend on your STL implementation. But once you figure out how it all works, then things become easy 😉 For exemple just to derivate a streambuf that just does nothing :

template <class cT, class traits = std::char_traits<cT> >
class basic_nullstreambuf: public std::basic_streambuf<cT, traits>
typename traits::int_type overflow(typename traits::int_type c)
return traits::not_eof(c); // indicate success

template <class _CharT, class _Traits>
basic_nullstreambuf<_CharT, _Traits>::basic_nullstreambuf()
: std::basic_streambuf<_CharT, _Traits>()

template <class _CharT, class _Traits>
basic_nullstreambuf<_CharT, _Traits>::~basic_nullstreambuf()

template <class cT, class traits = std::char_traits<cT> >
class basic_onullstream: public std::basic_ostream<cT, traits> {
std::basic_ios<cT, traits>(),
std::basic_ostream<cT, traits>(0),
basic_nullstreambuf<cT, traits> m_sbuf;

typedef basic_onullstream<char> onullstream;
typedef basic_onullstream<wchar_t> wonullstream;

Since AML, is aiming at being completely cross-platform, and that I started playing quite a lot with STL ( with the logger, but also with the delegate implementation written some time ago), I decided to use STLport. I am quite happy with it, as it comes along with extensive test code, and works fine on most platform. But sadly it lacks some documentation, help or activity around the community.

But to be honest I am actually quite impressed ( even myself ) at how easy it can be to add a new dependency check into wkcmake. Drop a usual FindModule.cmake in the WkCMake’s Modules (if it’s not already in CMake’s Modules ), add 3 lines to the WkDepends wrapper, one line to WkPlatform header, and that’s it. Next time you run cmake on your project you can see the detected includes and libraries for the new Module, and generate your project. I just did it for STLport on AML in a few minutes.

I am hoping I can get something stable for AML’s Core quickly, so I can start focusing on the network part, probably using ZeroMQ. Because multi player games are just much more fun, and after all, proper distributed simulation is what the project is also about since the beginning…

Categories: C++, CMake, WkCMake Tags: ,

SDL, OpenGL, PixelFormat and Textures

Between SDL and OpenGL, both have different ways of storing the Color of a pixel. SDL uses its PixelFormat as a structure providing information on how color information is stored, while OpenGL uses a list possible internal texture format when you create a new texture with glTexImage2D.

I struggled a little before being able to find a proper way to convert a SDL_Surface to an OpenGL texture recently for my graphic library, so I decided to post my findings here, because I couldn’t find any exhaustive information on the web easily… I will not talk about the uncommon palette image with transparency yet, or other 8bits displays 😉 Even nowadays with quite standard configuration there are some issues, and I will talk here only about displaying true color images on true color displays. If your image is not true color (that is pixel color information takes less than 24 bits ) you can always convert it externally as a data, or in your program.

  • First of all, if you want to care about OpenGL implementation earlier than 2.0, you need to convert your texture to a “power of two” size. that is width and height of 32 or 64, 128, 256, etc. To achieve that, in SDL, I create a new image with the same flags, and blit the old surface on the newly created one. If you care about transparency, you need to transfer colorkey and surface alpha values to the new surface before blitting, so that everything is blitted.
  • Then you have to convert your SDL_Surface to the proper format before using it as a texture. As your display will probably be true color if you are using OpenGL, you can use SDL_DisplayFormat to convert your surface to a > 24bits format. It seems possible to display palletized texture with OpenGL, but if you are not using a great amount of it, such a complicated technique can seem overkill. Also if you have a ColorKey on your surface you need to use SDL_DisplayFormatAlpha, so that the pixel with colorkey value will be converted to alpha (full transparent) on the new surface.
  • You need to find the proper texture format. Also there is an issue with the colorkey, which can sometime generate an alpha channel that you shouldn’t use (shown in SDL_PixelFormat with Aloss == 8). Here is the code I am using ( on a little endian machine )

    if (surface->format->BytesPerPixel == 4) // contains an alpha channel
    if (surface->format->Rshift == 24 && surface->format->Aloss == 0 ) textureFormat = GL_ABGR_EXT;
    else if ( surface->format->Rshift == 16 && surface->format->Aloss == 8 ) textureFormat = GL_BGRA;
    else if ( surface->format->Rshift == 16 && surface->format->Ashift == 24 ) textureFormat = GL_BGRA;
    else if ( surface->format->Rshift == 0 && surface->format->Ashift == 24 ) textureFormat = GL_RGBA;
    else throw std::logic_error("Pixel Format not recognized for GL display");
    else if (numbytes == 3) // no alpha channel
    if (surface->format->Rshift == 16 ) textureFormat = GL_BGR;
    else if ( surface->format->Rshift == 0 ) textureFormat = GL_RGB;
    else throw std::logic_error("Pixel Format not recognized for GL display");
    else throw std::logic_error("Pixel Format not recognized for GL display");
  • Once all this is done you can load your texture as usual with OpenGL, and everything should work 😉

    glGenTextures(1, &textureHandle);
    glBindTexture(GL_TEXTURE_2D, textureHandle);
    glTexImage2D(GL_TEXTURE_2D, 0, ptm_surf->format->BytesPerPixel, ptm_surf->w, ptm_surf->h, 0, textureFormat, GL_UNSIGNED_BYTE, ptm_surf->pixels);

I hope this little post will help those, like me, who like SDL and OpenGL, but have trouble mixing them. The code is available in my opensource graphic library, under heavey development at the moment. Our goal is pretty big, and help, in any way, is very welcome. Dont hesitate to contact us if you want to get involved 😉
And for the experts out there, let me know if you spot some errors in here.

Back to coding…

Categories: C++, OpenGL, SDL Tags: , , ,