View unanswered posts | View active topics It is currently Sat May 26, 2018 6:43 pm

Reply to topic  [ 4 posts ] 
 V/G shader to properly generate rectangle from 1-2 vertices 
Author Message

Joined: Sun Mar 08, 2009 7:24 pm
Posts: 103
Before I start, I want to say that I'm doing this to help StapleButter with BlargSNES. I'm attempting to develop a routine for displaying Mode7 using GPU acceleration, as it currently is handled via software-rendering (which is slow).

What I'm looking to do is take either 1 or 2 vertices I input, and output 2 polygons that form a rectangle. The rectangle itself is to have a width/height of 8 pixels (prior to matrix calculations), so in a case where I have 2 points, they would be on opposite corners of the shape, so the geometry shader would use them to generate the two other vertices to form an axis-aligned rectangle (using one's X and the other's Y). This is how BlargSNES currently handles tiles for modes other than Mode7, as they are axis-aligned. I am attempting to incorporate a model-view matrix that'll define the scale/rotation for a group of tiles (from either full-screen or per-scanline), but the results are not correct because I implemented it via the vertex shader.


The image itself is not rotating the tiles, but is instead squishing either horizontally or vertically because the 2 vertices are processed with model-view matrix calculations in the vertex shader (which now I know is done before the geometry shader, not after). I tried to shift such calculations to the geometry shader, but it didn't like that, resulting in a blank screen and eventually led to a crash.

Here's the code that would produce the results of the image above.

; setup constants
   .const c9, 0.0, 0.0, 0.0078125, 1.0
; setup outmap
   .out o0, result.position
   .out o1, result.color
   .out o2, result.texcoord0
; setup uniform map (not required)
   .uniform c0, c3, projMtx
   .vsh vmain, end_vmain
   .gsh gmain, end_gmain
; v0: vertex (x, y, and optional z)
; v1: texcoord
; c0-c3: projection matrix
; c4: texcoord scale
      mov r2, v0 (0x4)
      mov r2, c9 (0x3)

      ; my inclusion of modelview-matrix calculation, which this (and projMtx) may need to be in the geometry shader if possible?
      dp4 r1, c5, r2 (0x0)
      dp4 r1, c6, r2 (0x1)
      dp4 r1, c7, r2 (0x2)
      dp4 r1, c8, r2 (0x3)

      ; result.pos = projMtx * temp.pos
      dp4 o0, c0, r1 (0x0)
      dp4 o0, c1, r1 (0x1)
      dp4 o0, c2, r1 (0x2)
      dp4 o0, c3, r1 (0x3)

      ; result.texcoord = in.texcoord * (uniform scale in c4)
      mul r1, c4, v1 (0x5)
      mov o1, r1 (0x5)
      ; turn two vertices into a rectangle
      ; setemit: vtxid, primemit, winding
      ; v0 = vertex 0, position
      ; v1 = vertex 0, texcoord
      ; v2 = vertex 1, position
      ; v3 = vertex 1, texcoord
      ; x1 y1
      setemit vtx0, false, false
      mov o0, v0 (0x5)
      mov o1, c9 (0x8)
      mov o2, v1 (0x5)
      ; x2 y1
      setemit vtx1, false, false
      mov o0, v2 (0x6)
      mov o0, v0 (0x7)
      mov o1, c9 (0x8)
      mov o2, v3 (0x6)
      mov o2, v1 (0x7)
      ; x1 y2
      setemit vtx2, true, false
      mov o0, v0 (0x6)
      mov o0, v2 (0x7)
      mov o1, c9 (0x8)
      mov o2, v1 (0x6)
      mov o2, v3 (0x7)
      ; x2 y2
      setemit vtx0, true, true
      mov o0, v2 (0x5)
      mov o1, c9 (0x8)
      mov o2, v3 (0x5)
;operand descriptors
   .opdesc x___, xyzw, xyzw ; 0x0
   .opdesc _y__, xyzw, xyzw ; 0x1
   .opdesc __z_, xyzw, xyzw ; 0x2
   .opdesc ___w, xyzw, xyzw ; 0x3
   .opdesc xyz_, xyzw, xyzw ; 0x4
   .opdesc xyzw, xyzw, xyzw ; 0x5
   .opdesc x_zw, xyzw, xyzw ; 0x6
   .opdesc _y__, yyyw, xyzw ; 0x7
   .opdesc xyzw, wwww, wwww ; 0x8

I'm still in the early stages of learning shaders, so while I have some concepts down, there is still a lot I don't understand, and this is one of them. If all goes well, I'd attempt to do this with only one vertex input and have the geometry shader generate 3 additional vertices, as the width/height never change.

Sat Mar 07, 2015 7:37 pm

Joined: Thu Feb 03, 2011 10:47 pm
Posts: 210
This approach is simply not going to work for affine transformations. Your geoshader assumes that the two added vertices are axis-aligned with the two input vertices, which is simply not true for affine transformations (except for obvious cases like 90 degree rotation or horizontal flip, among others). You need to pass the affine matrix to the geoshader and calculate the missing vertices with this transformation.

The problem I see you immediately running into is that many games use Mode 7 to create a perspective transformation by adjusting the affine matrix per-scanline. You will probably have to rely on hacks that know a game is creating a certain perspective via scanline tricks and use a pre-calculated perspective transform.

An alternative is to have the emulator calculate all of the vertices and pass them all to the vertex shader, ditching the geoshader altogether. Of course, you can take this approach for only Mode 7 backgrounds and still use your geoshader approach for the other backgrounds since they will be axis-aligned.

Sun Mar 08, 2015 4:31 am

Joined: Sun Mar 08, 2009 7:24 pm
Posts: 103
Yeah, I started to see that as I kept trying and trying. I resorted to throwing in 4 vertices (and figured out why it wasn't working before). I understand that there may be problems with per-scanline sets, but I'll tackle that when I get to it. First I want to get full-screen, unmodified-scanline Mode7 working.

So far everything looks good, except I can't get the positioning for the layer correct. I had to use the inverse of the affine matrix to get the scaling/orientation correct, but getting the position correct is lost to me atm.

Sun Mar 08, 2015 5:57 am

Joined: Sun Mar 08, 2009 7:24 pm
Posts: 103
Ok, I believe I got the calculations for positioning correct. In much the same way I had to calculate the right affine matrix (by getting it's inverse), I sorta did the reverse on calculating the position. Now I can work on figuring out which tiles are to be rendered, which if I've thought this out carefully, would be kinda like rendering a flat-shaded polygon in software, but mapping the routine to the layer to grab the right tiles.

Sun Mar 08, 2015 8:50 pm
Display posts from previous:  Sort by  
Reply to topic   [ 4 posts ] 

Who is online

Users browsing this forum: No registered users and 0 guests

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
  Powered by phpBB® Forum Software © phpBB Group
Designed by ST Software.
Get devkitPro at Fast, secure and Free Open Source software downloads