Page 1 of 1

Some questions about endianness and rgb (Quake)

Posted: Sat Apr 20, 2024 4:55 pm
by yummt
Hello,

I've been working on the QuakeGX port and ported over some features from better engines such as HLBSP, Skybox, Fog(I have a lot of questions about fog but I'll save that for later) and now I'm working on porting colored lighting.

https://github.com/tyleryoung88/QuakeGX-Wii

It's been a process for me to port over these features, and along the way I encountered dealing with endianness of the Wii, I have never worked with a Big Endian machine so this was new to me.

It started with half life textures. At first they were red, I eventually realized the RGB values were flipped. So after flipping the rgb values the textures loaded properly. Then encountered the same problem with skyboxes. Every image I loaded into system memory had flipped RGB values. I was able to fix this by flipping the byte order of the entire image after it was loaded into memory.

Now, I'm working on porting .lit support to the engine, and expanding the white light to 3 color RGB. Behold, when I load a lit file and expand the color, the RGB values are flipped yet again.
However, this time I'm not able to flip them back for some reason. I've tried to reverse the byte order of the lit data,but that didn't do anything. I can't seem to extract the correct RGB values from the data, and I'm not sure where to begin. HalfLife lightmaps are also loading into memory incorrectly. I'm not sure if it's related to all of this.

My question is, why is this happening? Is it because of the endianness of the machine, or is it that Wii reads RGB as BGR somewhere in the pipeline?

Has anyone else encountered these issues? I can't find much of anything on the internet. I've researched the Wii as much as I possibly can, used multiple search engines, even found the sdk kit and read through documentation and examples there. At this point I'm stumped and just hoping someone else could have some more understanding of what's going on here. What gives..?

Thank you for taking the time to read this! Cheers.

Re: Some questions about endianness and rgb (Quake)

Posted: Mon Jul 01, 2024 7:51 pm
by barfoo34
yummt wrote: Sat Apr 20, 2024 4:55 pm I've been working on the QuakeGX port and ported over some features from better engines such as HLBSP, Skybox, Fog(I have a lot of questions about fog but I'll save that for later) and now I'm working on porting colored lighting.
Nice! For fog, you might want to have a look at opengx, basic fogging is working there (tested with the gl-117 game and other simple OpenGL samples).
My question is, why is this happening? Is it because of the endianness of the machine, or is it that Wii reads RGB as BGR somewhere in the pipeline?
It might be an endianness issue, but in that case it's usually quite easy to find out. I had a look at your code, and in particular at the GL_Upload32() function. It operates on single bytes, and at a first glance it seems correct. But it expects pixels to be layed out as ABGR, which is a bit unusual; so, I'd check if the rest of your code is preparing the texture data in ABGR format.

You might want to look at different implementations: in opengx I'm using a templated class (TexelRGBA8 in src/pixels.cpp), whereas in SDL 2.0 you can look at a pure C implementation, which supports both RGBA and ARGB (but not ABGR!) formats:
https://github.com/devkitPro/SDL/blob/o ... gcpixels.c