Skip to content

Preserve full float precision in strands shader literals - #9408

Open
harshiltewari2004 wants to merge 1 commit into
processing:mainfrom
harshiltewari2004:fix-strands-float-literal-precision
Open

harshiltewari2004 wants to merge 1 commit into
processing:mainfrom
harshiltewari2004:fix-strands-float-literal-precision

Conversation

@harshiltewari2004

Copy link
Copy Markdown
Contributor

Resolves #8884

Changes:

  • Float literals in strands shaders were serialized with toFixed(4), so small values like 0.00001 became 0.0000 and others were rounded. Both backends now use a shared formatFloatLiteral() helper (in ir_types.js), following @bhabishnu's suggestion on the issue: Number.isInteger(v) ? v.toFixed(1) : String(v). This keeps full precision, keeps integer values in float form (1.0), and uses exponent form only at extreme magnitudes, which both GLSL and WGSL accept.
  • Added a test that checks the generated shader source keeps 0.00001 and 0.123456 intact.

Out of scope: @bhabishnu also noted that raw numeric arguments (e.g. abs(3000000000)) go through a separate path and differ between WebGL and WebGPU. Happy to follow up on that separately.

Note: handle passing a value from a vertex hook to a fragment hook with swizzle assignment in test/unit/webgpu/p5.Shader.js fails locally for me on current main both with and without this change, so it looks pre-existing and unrelated.

PR Checklist

  • npm run lint passes
  • [Inline reference] is included / updated
  • [Unit tests] are included / updated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[p5.js 2.0+ Bug Report]: Floats in strands shaders are always rounded to 4 decimals

1 participant