-
Notifications
You must be signed in to change notification settings - Fork 407
OSL Network Generator updates #2739
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ld-kerley
wants to merge
2
commits into
AcademySoftwareFoundation:main
Choose a base branch
from
ld-kerley:osl/network-shadergen-parity
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
OSL Network Generator updates #2739
ld-kerley
wants to merge
2
commits into
AcademySoftwareFoundation:main
from
ld-kerley:osl/network-shadergen-parity
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…tor. This is a pretty large commit, but most of it is all pretty intertwined, so I'll do my best to outline here all the different changes. ### LibsToOso * The tool now supports selecting using the NodeDef name or the Implementation name as the basis for the name of the new OSL Network generator implementation and corresponding OSO. * OSL code and corresponding OSO is now generated directly from the NodeDef element. * `implname` functionality added is used to define the renamed ports. * NodeGraph elements can now be skipped during processing. ### OSLNetworkGenerator * Generator now directly inherits from `ShaderGenerator` as there is no common functionality shared with the `OSLShaderGenerator`. * Provides API to compile OSL source, and also to generate a Shader from a NodeDef element, this is consolidated from the `LibsToOso` tool source. * The generator can now create a new ShaderNodeImpl object on demand (and the corresponding compiled OSO file) if a locally defined OSL Source Implementation element is discovered. This includes correctly constructing an OSL Source generator object with all the corresponding custom types defined to allow for the correct shader generation to occur. * `liboslcomp` can be optionally used for the on-demand compilation for better performance, and avoids writing the shader source to disk to compile. * New `ShaderGraph::flattenGraph()` function is used to correctly expand any locally defined NodeGraph elements in to the corresponding nodes. We do not on-demand compile non-functional NodeGraph elements in to OSOs as they do not have corresponding NodeDef elements to fully name and decorate the interface. Once the `ShaderGraph` is flattened then we are left with a fully expanded graph of individual nodes. * `OslNetworkSyntax` has been refactored. * It no longer has any reserved words, as the grammar used to describe the network has no restrictions on words that are used. * Parameter value emitting code has been refactored to better support the "built-in" struct types more cleanly, and removes dependency on the OSL Source syntax, as they are fundamentally two different exports. Including better error handling for types that do not have direct parameter export. ### MaterialXGenShader * New API method `GenContext::getSourceCodeSearchPath()`, needed to allow dynamic construction of derived generators that share the source code search path. * New `GenOptions` option `oslTempOsoPath` to allow control over where on-demand OSOs are compiled to. If unspecified the system will fallback to a uniquely generated temp directory in the system temporary directory. * Support added for `implname` attribute during shader generation. This is described in the specification, but not yet implemented. During OSL source generation when compiling the OSOs its possible input names such as `normal` may be modified to avoid clashes with reserved words. Adding `implname` support allows this to be described in the generated Implementation elements and respected during OSL Network generation. New `getPortName()` function is added in support for the implementation of this feature. This is also needs to be stored in two separate classes, the actual implementation can be either a `ShaderGraph` or a `ShaderNodeImpl`. * New API method `ShaderGraph::flattenGraph()` will walk the graph and expand any `CompoundNode` entities discovered. Care is taken to generate unique names for newly generated nodes by prefixing the names with the name of the prior parent `CompoundNode`, and also to ensure all connections are retained. * `ShaderGraph::create()` can now accept and populate the created graph directly from a NodeDef element, which is simpler than the current Node source, as there are no local modifications. * `Syntax` base class now tracks custom registered syntaxes so they can be easily queried. This is necessary to support dynamic creation of a derived generator, specifically to allow the OSL Network generator to create the OSL Source generator for on-demand compilation of locally defined nodes. Other supporting changes are outlined below. ### Cmake * Add CMake option to control if NodeGraph implementations are compiled down to OSOs, or left as NodeGraphs. ### MaterialXFormat * Add optional `recursive` argument to FilePath::createDirectory(), to allow for recursive directory creation if necessary. * Add API to FilePath to return the system temporary directory, as well as create a new temporary directory. This is used for on-the-fly compilation of OSL source implementation to OSOs if defined locally in a document and not in a pre-compiled standard library. * Add a new `createDirectories` option to XmlWriteOptions to allow for any necessary directories to be created while writing a new file. ### MaterialXTest/MaterialXRenderOsl * Add new test specific implementations for tangent and bitangent nodes for the OSL Network generator. These may become obsolete if the OSL render testing moves to using the OBJ file (see XXXX). * Update the OSL Network scene template to align the cameras after the OSL Source template was changed. * Remove the OSL Network specific tests that were skipped - we now have complete one-to-one parity with OSL Source generator. * Write fully expanded paths in to the template file for easier testing. * Introduce string processing to the command string that needs to be embedded in a legal XML file to replace illegal XML characters `<` and `>`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Update OSL Network generator to provide parity with OSL Source generator.
This is a pretty large commit, but most of it is all pretty intertwined, so I'll do my best to outline here all the different changes.
LibsToOso
implnamefunctionality added is used to define the renamed ports.OSLNetworkGenerator
ShaderGeneratoras there is no common functionality shared with theOSLShaderGenerator.LibsToOsotool source.liboslcompcan be optionally used for the on-demand compilation for better performance, and avoids writing the shader source to disk to compile.ShaderGraph::flattenGraph()function is used to correctly expand any locally defined NodeGraph elements in to the corresponding nodes. We do not on-demand compile non-functional NodeGraph elements in to OSOs as they do not have corresponding NodeDef elements to fully name and decorate the interface. Once theShaderGraphis flattened then we are left with a fully expanded graph of individual nodes.OslNetworkSyntaxhas been refactored.MaterialXGenShader
GenContext::getSourceCodeSearchPath(), needed to allow dynamic construction of derived generators that share the source code search path.GenOptionsoptionoslTempOsoPathto allow control over where on-demand OSOs are compiled to. If unspecified the system will fallback to a uniquely generated temp directory in the system temporary directory.implnameattribute during shader generation. This is described in the specification, but not yet implemented. During OSL source generation when compiling the OSOs its possible input names such asnormalmay be modified to avoid clashes with reserved words. Addingimplnamesupport allows this to be described in the generated Implementation elements and respected during OSL Network generation. NewgetPortName()function is added in support for the implementation of this feature. This is also needs to be stored in two separate classes, the actual implementation can be either aShaderGraphor aShaderNodeImpl.ShaderGraph::flattenGraph()will walk the graph and expand anyCompoundNodeentities discovered. Care is taken to generate unique names for newly generated nodes by prefixing the names with the name of the prior parentCompoundNode, and also to ensure all connections are retained.ShaderGraph::create()can now accept and populate the created graph directly from a NodeDef element, which is simpler than the current Node source, as there are no local modifications.Syntaxbase class now tracks custom registered syntaxes so they can be easily queried. This is necessary to support dynamic creation of a derived generator, specifically to allow the OSL Network generator to create the OSL Source generator for on-demand compilation of locally defined nodes.Other supporting changes are outlined below.
Cmake
MaterialXFormat
recursiveargument to FilePath::createDirectory(), to allow for recursive directory creation if necessary.createDirectoriesoption to XmlWriteOptions to allow for any necessary directories to be created while writing a new file.MaterialXTest/MaterialXRenderOsl
<and>.