My first 2 cents

[visualart]

Hi, new user here some might be a couple of things in here that are allready covered due to my inexperience with the program

1. Motionblur: control over the amount of interpolation steps
2. DOF: currently looks like unuseable due black outline artefacts and bad box blur (looks more like a bad glow then an actual depth blur
3. Overall alpha black outline artefacts, looks like there's some misinterpretation of premultiplied vs straight alphas. So far i get better results with straight alphas rather then premultiplied but then again when applying the texture to something with depth like particle sprites the artefacts are introduced again.
4. A basic relighting tool would be handy, based on normals and zdepth ref. Normality plugin for AE.
5. Support for multichannel EXR format, especially towards multilight renderpass rendering, ref. Maxwell renderer plugin for AE.
6. 3D Alembic file import. (much faster then fbx, support for pointcloud and baked animation as well)
7. Deep compositing. There's a little workaround for that using a plane with as many vertexpoints as pixels and displacing based on the z-depth pass in an ortographic cameraview.
8. Kinect zbuffer feed (could workaround it with spout and processing)
9. Parameterized object instancing and replicator tool with offset control over time (like being able to control delay and multiply values)
10. UI wise, having a nodebased particleview would clarify the workflow a bit more, especially when you start having more complex setups and more conditionbased modifiers would be handy.
11. A match aspect from image source for texture layers
12. A simple link to another layer like the pickwhip from AE
13. Basic expressions on parameters ref. AE expressions or MAX wire parameters.

that's it for now, i'll propably come across other things while using

[francis]

Hi Tom,

Thanks for the feedback! If I understand well, we're going to discuss this very soon

Francis.

[visualart]

Hi Francis, you can allready put the bubbles in the fridge, I think I'll be there around 16h

[martin]

Hey Tom !

DOF: currently looks like unuseable due black outline artefacts and bad box blur (looks more like a bad glow then an actual depth blur

Yeah this is an historical bug with the blur and how it acts with (premultiplied) alpha. Try a compo 3D box and a blur over it, and you'll see the results are vastly different with or without an alpha layer:

Blur on image without alpha -

Blur on image with alpha -

It's too bad cause the DOF always worked but has been unused because of this artefact. Good to see some feedback on it ! Maybe the historical Blur vs. Alpha bug will get prioritized a bit more thanks to this

---

Parameterized object instancing and replicator tool with offset control over time (like being able to control delay and multiply values)

There is an internal tool that exists to do that, but is not available to the public due to lack of time to develop it, called the Instantiator. It allows you to create instances of a shared element over time, and would eventually allow to to interpolate some of the (exposed or not) parameters of the different instances over their instance indices.

If you have examples of tools or plugins in other software that allow you to do that, please share them ! If you have scenarios / use cases for such a tool, please share them too.

Perhaps it would be a good occasion to bring the Instantiator to the public !

---

A simple link to another layer like the pickwhip from AE

There is no tool like the pickwhip, but you could do that today with groups (although I agree, it's not a direct layer to layer placement dependency).

---

Basic expressions on parameters ref. AE expressions or MAX wire parameters.

What could exist for that is a LUA link modifier. You could create a local link (modified by a Script Link Modifier) inside your compo, obtaining the equivalent of MAX wire parameters. What we have (still internal because of lack of demand on it) is a LUA script tool, working with local exposed parameters (which also does what you ask here).

[francis]

Some remarks from my side:
@vincent.armongn is welcome to join discussion, but probably at some point we should spread this into multiple different topics...

(3.) As I told you yesterday, for the moment you need to order your 3d layers from back to front manually. For particles, you also need to enable the "sort particles" option to remove artifacts. Is that enough to fix your problem or do you have a specific image/scenario in which there is a premultiplied vs. straight mismatch? If you have a particular image that fails, can you send it to me?

(4.) I just created an issue for this in our system. You mention normal maps, but isn't it enough to have the depth? In principle, we can recreate normals based on the depth map only (although they may present more artifacts than a native normal map). What do you think, should we require both?

(5.) I also created an issue for this in our system. Could you send us a sample EXR file with multilight rendering passes?

(6.) We used to support Alembic in the past, but only for the first frame. So here we need to implement two things: support of Alembic + "geometry streaming" engine. I created an issue for it in our system.

(7.) Did you try to workaround? does it work? I think first, we should do EXR support... (5.). I see two different levels here:
- the "easy way": add EXR support and support Z-test when rendering the 2d layers
- the "hard way": store fragment-lists for each pixel and do the rendering in a defered way to guarantee correct blending independently of the order.
(The hard way would take much longer!!)

(8.) It's already a topic we are discussing, there is an issue in our system, but it's not yet planned.

[visualart]

Hi francis,

some quick reply on 4, i'll take some time tomorrow to create some more details and images for the others, with the normals you'll have more detail in the relighting as for example you can render out a normal bump pass in 3d which takes into account bumpmapping.

for 7 i'll try to make an example in SMODE, in case EXR's are too heavy in a realtime enviroment, it could be an option to seperate the passes into other file formats and load them in automaticly by using a fix naming convention.

And thanks for the evening to all of you by the way, we had a blast, we should do that again soon !

[visualart]

About the deepcompositing trick, i think it's quite an interesting approach what i have so far,
here's the compo : https://www.dropbox.com/s/2lt525p358y4oty/deepcompo.zip?dl=0

I've putten in the scene some relighting and animated depth smoke. What I didnt get to work is being able to assign coloured points based on the original image so I used normal mesh displacement.

So what would be an ideal workflow now is to have a seperate deepcompo layer for example where you have following parameters:

1. diffuse colour map
2. specular map (for relighting you don't want every pixel to be highlighted the same way)
3. z-depth map
4. alpha map (could be inherited from the diffuse colour map)
5. displacement amount slider or value
6. In case you're relighting, you should be able to choose the blending mode on how the relighting is applied

one thing that needs to be adressed as well is a light feathering or antialiasing of the points as you'll get some pretty harsh edges.

[francis]

Ooh, that's interesting!

I think you forgot the normal map in your enumeration, no?

What I didnt get to work is being able to assign coloured points

Normally, the color is what you put inside the "Diffuse" component.
Are you talking about Modifiers > Attributes > Colorize Layer ? (this is the modifier to colorize on a per-vertex basis)

antialiasing of the points as you'll get some pretty harsh edges.

For the aliasing, short term solution is to use a post effect Blur > AntiAliasing (it's FXAA).

But indeed, the whole thing would be more clear (and artifacts more controllable) if directly computed on a per-pixel basis (with dedicated new shaders) instead of having this huge geometry.

Could you send an example or two of {diffuse,specular,z,normal,alpha} test map sets? that would help
Especially:
- one simple normalized image of, e.g. a sphere of radius=1 seen by a camera of fov 60° at distance=5m
- two deep images that are intended to be merged

I think that it should be possible to do something for that next week, but for the moment I have some troubles finding good sample images on the net.

[visualart]

Hey Francis, glad you like,

I was talking about when using sprite or point renderer if I put an image as source, it distributes the image to the particles seperatly instead of assigning each pixelcolour to the according particle.

For this deepcompositing, you actually won't need the normalmap, the normalmap is for the other 2.5D relighting technique ref. normality for AE, this one is going to be less accurate then using the deepcompo trick but for me it's a quick fix trick i use a lot in aftereffects to add some minor lightingfx on mattepaints. The deepcompo trick doesnt work that well in AE as you can't use ortographic cameras there. Another map I forgot to mention in this is Matcap maps as well and screenspace ambient occlusion (i know you allready have proper AO in there but it's too slow for this, screenspace AO might speed things up a bit), here's a ref to a plugin in AE that might give you some ideas as well:

Realtime SSAO with After Effects and Pixel Bender

[visualart]

this is also quite a good reference as workflow : http://aescripts.com/pixel-cloud/

[francis]

I was talking about when using sprite or point renderer if I put an image as source, it distributes the image to the particles seperatly instead of assigning each pixelcolour to the according particle.

yep, this you can do it with Modifiers > Attributes > Colorize Layer (it will assign one color to each sprite).

talking to tim right now, will do longer answer later ...

[francis]

For this deepcompositing, you actually won't need the normalmap

indeed, but I'd like to avoid to have to use the Plane+Displace+SurfaceComponents deepcompositing trick and instead promote a native concept of "Deep Layer" in Smode with a serie of dedicated rendering shaders.
The challenge is thus to find some concepts to unify all this (be it relighting with fixed viewpoint or 3d point clouds with 3d camera). The concept of "depth" already exists in our todays pipeline of 2d layers, but for the moment it's only implemented for the 3d generated by smode inside pre-compos. So, my overall idea is to add some elements to specify depth from an image + add new renderers "DeepImagePointCloud" and "DeepImageWithDepthTest" + a new modifier "RelightDeepImage".
For the normals: I guess simply, we should support two modes for the normals in our deep layers: implicit (deduced from the depth map) or explicit (with a dedicated map).

Matcap maps

Is that the same as spherical environment maps? Can you provide an example?

i know you allready have proper AO in there but it's too slow for this, screenspace AO might speed things up a bit

In fact the only implementation of AO we have is SSAO and it relies on the depth I mentioned above.
(to use it: on a 2d layer resulting from a 3d precomp: **Modifier > Stylize > Ambient Occlusion").

and please send me sample depth/diffuse/specular/normal textures I can play with

[visualart]

Here we are again, doing a big project so running into some things again

• First biggest issue would be, and you allready told you're changing but it's the license system, the SMODE synth is cute to use but not being able to import videos as layers is kinda like selling a car without wheels imho. You'd be better of with a fully functional SMODESynth without being able to output. On the project now i'm introducing freelancers to SMODE but I constantly have to say like yes but in studio it's working and they can't test.

• Make a difference in Compo and for example PreCompo. Now we're working on the HD version to produce the visuals, when using a (pre)compo as a UVwrapped texture, there's a limit to the resolution and watermark appears. In this 1 scene i need for example a 8k+ baked uvmap where i selectively animate some parts within smode on the texture in a compo. Reducing that map to 1200 pixels is not an option. Differentatiating between main comp and precomp without limitations could solve that

• The fbx update doesnt seem to work good, when an object is updated nothing happens in SMODE

• A global asset manager like in 3dsmax would be handy, like being able to easily replace used footage without having to dig into the complex compos.

• The depth of field is really an issue, it's simply totally unusable

• For the AO, maybe a baking solution might be an option ? As it is now it's a big performance drop.

• For the volume lights make an exclude light option list, usually you don't want all the lights visible

• more shadows for different light types.

• A long shot propably but are VOXELS an option to implement ? you allready got a decent particle system, would be a nice addition.

That's it for now, i'll propably get to more stuff along the way

grtz : T

[francis]

Hello Tom,

I answered the two first points by email. Some hints for the other points here:

Fbx update: did you try "right click on the FBX > Update File"? Which version are you using? (we fixed something about this two weeks ago).

Global asset manager: that would indeed by nice

Depth of field: to ease it's configuration, you can enable option "Visualize Layers" to first properly setup the depth settings. Did you try this?

AO: if you are doing camera mapping with a fixed viewpoint then you can bake it with e.g. C4D.

Volumetric: in the advanced parameters of Light you can access to Volumetric - Enable / Level to exclude some lights from the volumetric light calculation.

More shadows: needs a bit of time, but yes totally

Voxels: would be nice, so many things to do!!!

Maybe @vincent.armongn or @Martin can complement...

Best,

Francis.