Page 1 of 1

C++ stream function lockup libFAT

Posted: Wed Dec 30, 2009 10:43 pm
by jotd
Hi,

I just want to report this (on devkitpro 1.5):

I was using isstrstream to parse integers, floats instead of atoi/atof. Worked OK with NitroFS on emus, but on the real DS with libFAT it locks up.
I don't see why it locks up since even if it's a stream, it's a stream on a buffer, not on a file, so it's strange that the I/O functions get in the way here.
I replaced my parsing of istrstream by atoi and everything worked fine.

I'm sure it's the cause of the problem, because I had to do it twice on 2 different code locations, because I had 2 sources of freezes.

Re: C++ stream function lockup libFAT

Posted: Thu Dec 31, 2009 12:21 am
by WinterMute
Unfortunately emulators will quite often run code quite happily that will then die on hardware. It's difficult to say what the problem might be without seeing your code but STL streams are quite memory heavy - see http://www.coranac.com/2009/02/some-int ... code-size/ for some numbers.

It's likely that your NitroFS code will also fail on hardware - you can test this using the homebrew menu as a launcher.

Re: C++ stream function lockup libFAT

Posted: Thu Dec 31, 2009 9:04 am
by jotd
WinterMute wrote:Unfortunately emulators will quite often run code quite happily that will then die on hardware. It's difficult to say what the problem might be without seeing your code but STL streams are quite memory heavy - see http://www.coranac.com/2009/02/some-int ... code-size/ for some numbers.
I avoid streams as much as possible. There were some remaining code that caused problems. I'll code some small reproducer for you.

About memory concerns, adding ffunction-sections and -Wl,--gc-sections helps (see my post in "suggestions"):
http://forums.devkitpro.org/viewtopic.php?f=14&t=1665
WinterMute wrote: It's likely that your NitroFS code will also fail on hardware - you can test this using the homebrew menu as a launcher.
Err, no I can't because it freezes (see my post: homebrew menu freezes :)) http://forums.devkitpro.org/viewtopic.php?f=15&t=1668

I shall do what Steve advised again. May improve something.

Re: C++ stream function lockup libFAT

Posted: Thu Dec 31, 2009 4:00 pm
by StevenH
jotd wrote:About memory concerns, adding ffunction-sections and -Wl,--gc-sections helps (see my post in "suggestions"):
http://forums.devkitpro.org/viewtopic.php?f=14&t=1665
Your confusing the file size of the nds file and the memory footprint of STL - it's 2 separate things. Yes you will be removing unused functions and reducing the nds file size, BUT the way STL works is not as simple as you would think.

your istrstream class uses a few extra bytes of memory to store a pointer to the base of the class structure in memory, a pointer to a function list and then extra bytes for every variable that the class uses that is defined in the class header, then you also have the class functions (even if your not using them they are still added as they are part of the class which can not be reduced in size due to the way that STL code works), and then you need to include the base class functions, and it's base class, so there's not a lot of code that you can remove. After adding all these extra bytes together means that re-writing your code to use pure C atoi and atof, or even the siscanf you will save way more memory than you think, and I mean memory not file size people keep thinking that file size == memory usage it's not since they are 2 different things.

Re: C++ stream function lockup libFAT

Posted: Thu Dec 31, 2009 6:04 pm
by jotd
StevenH wrote:
jotd wrote:About memory concerns, adding ffunction-sections and -Wl,--gc-sections helps (see my post in "suggestions"):
http://forums.devkitpro.org/viewtopic.php?f=14&t=1665
Your confusing the file size of the nds file and the memory footprint of STL - it's 2 separate things. Yes you will be removing unused functions and reducing the nds file size, BUT the way STL works is not as simple as you would think.

your istrstream class uses a few extra bytes of memory to store a pointer to the base of the class structure in memory, a pointer to a function list and then extra bytes for every variable that the class uses that is defined in the class header, then you also have the class functions (even if your not using them they are still added as they are part of the class which can not be reduced in size due to the way that STL code works), and then you need to include the base class functions, and it's base class, so there's not a lot of code that you can remove. After adding all these extra bytes together means that re-writing your code to use pure C atoi and atof, or even the siscanf you will save way more memory than you think, and I mean memory not file size people keep thinking that file size == memory usage it's not since they are 2 different things.
I mean ROM code. Of course, class instantiation cannot be guessed from the size of the .elf. But as all .text & .ro sections are finally loaded in RAM, removing such functions still save some memory.

And if a template function is not called, the code is not even generated (you can have syntax errors in functions, it is not detected until you actually call them). You're right, I'm no STL expert and I had problems with it in the past, also it sucks because of the history of it and the non-integration in the C++ language, the unintelligible memfunref shit that 0.01% of the C++ coders can write from memory, etc... Java is much more integrated (but sucks on the DS :))

I'll try to get rid of all C++ stream templates. They're not very useful, and all my code uses a middleware layer I have developped so the effort is not that important.

thanks