Is time-displacement effect possible with Smode?

[illtellyoulater]

Hello!
I know Smode uses ISF shaders and that there is a "slit-scan" ISF preset, but I am having trouble re-creating this very popular time displacement effect which has been recently used a lot on socials: https://www.toolfarm.com/tutorial/in-depth-time-effects-time-displacement-aka-slit-scan-with-examples/

Could anyone help me with this? I tried combining slit-scan.isf with other effects but I could't get something that matches the effect I am interested in.
I would really love if someone could prove Smode is capable of this effect... but I am about to give up. Admitting that Smode can't do this would be very sad, because so long it has been a pleasure using it...

So... looking forward to your wise thoughts!

[Walter_heen]

Hello!
I have no problem doing this effects with the feedback modifier.
Here is my layer tree:

Compo

• video input

• FeedBack modifier( blending normal)

  • PlacementMask -> Height (around 1px, moove the Y axis of the mask)

Is it working for you ?

[illtellyoulater]

Is it working for you ?

Actually no, it's not doing much. With the settings you suggest all I see is this (not sure if I am doing anything wrong):

This is much different from the results I'm getting with After Effects using the time-displacement effect (see it at: https://streamable.com/ybe9x6)

There's also this example https://youtu.be/fyJ2RERuAoo?t=25 (at 00:25) which is a bit more extreme but it's the same effect.

Do you think it's possibile? Thanks a lot!!!

[Walter_heen]

Hello!
here is the composition i suggest you to use for making a time displacement:
TimeDisplacement.compo (31.8 KB)

You only have to replace the Video input device by yours

Cheers

[illtellyoulater]

Hi again Walter!

Thanks a lot for your example, now I understand what you said!
However the result is a bit different from what I get in After Effects... Look for yourself, here is my After Effects output video https://streamable.com/ybe9x6
In After Effects it seems that the movement effect is continuous on all parts of the footage at the same time, and not just based on a single moving mask moving vertically along the picture.
Do you think the compo can be modified to reflect this different behaviour?
I don't want to beg for your help but I am quite new to compositing so I really have no idea where to look for...
I tried adding more placement masks at 0%, 25%, 50% and 75%, running down in sync together, but again the result is different from After Effects, where the effects is more "fluid" and looks way nicer.
Again, I appreciate a lot your precious help!

[Walter_heen]

The magic of the feedback modifier is that it accept other modifier in it.
Apply a 2d Transform to it with tiny position offset (maybe a backNForth loop with very small offset) ?

[illtellyoulater]

Hi Walter,
I tried to do what you said: I added a 2D transform to the feedback modifier (which already contained a placement mask as you suggested before), and then I animated the X position of the 2D transform.
However the result is definitely different from what I am trying to do...
Below I am sharing a link to my compo and a link to the original video file I am using (so, if you want, you can compare our Smode results to the After Effects results I previously shared).

compo: https://drive.google.com/file/d/1PoA5C2hfXghN2s6EN4yHpNEQ11oObmoa/view?usp=sharing
original video file: https://drive.google.com/file/d/1ooaWaZvyowAfIQivMcc0BTFbeOPGZ3j8/view?usp=sharing

I hope you can help me here... I'm really trying to use this use-case to learn more about Smode, but maybe I picked a really hard example? ...
Or... could it be that this effect is not possible at all using Smode?
I really want to prove that this is possibile, because I'd like to keep using Smode instead of having to accept that other software (e.g. After Effects) is more powerful when it comes to certain effects...
I am putting my faith in you !
Thank you again!

[illtellyoulater]

Anyone?

@alex
@Walter_heen

[Walter_heen]

Hello!
Sorry for the late answer.
I think that you only can simulate it with the feedback technics I advise you to use. Or wit hthe ISF shader.
The thing is that this effects need to keep x frames in memory. I'm affraid that this is not a Realtime rendering logics.

[illtellyoulater]

Thanks @Walter_heen, I understand, so there's no hope that Utility>Freeze or Utility>FrameDelay could be of any help here? Or what about getting the old frames by rewinding the video and replaying it on certain conditions?
Yes you could say "go on and try it" but unfortunately I don't master Smode enough to be able to fully wrap my mind around the technical realization of that.
If what I said is remotely possibile, could you please give it a try?
Thanks so much! Appreciate your help!

[Walter_heen]

I have tried, and the problem is that Smode, with it's realtime rendering engine, has no way to keep x frame in memory.
So rewinding in the realtime is complicated.

As you can see, the frameDelay modifier has limits in the delay. This is because we cannot keep frames in memory without creating memory issues.

That's for a realtime workflow.

For a video workflow, I'll have some ideas, but it requires too many time to test it at the office. But for what I guess, it would works with A LOT of timelines, all connected and offsetted in time through a function, for each line and/or column of pixel that compose your video. And even with this idea, i'm not quite sure of the result.

Python scripting could be a way to automatize it. But there is few chances that this kind of composition would run smoothly in realtime

Message written while thinking about it

[illtellyoulater]

@Walter_heen I see, thanks a lot for your detailed answer!

Last question (I promise! ): so do you think Adobe After Effects can achieve a wider range of results concerning effects, compared to Smode?

I'm just trying to understand the points of strength of both applications to see on which one I should focus more in order to explore visual effects creation (real-time or not).

[Walter_heen]

For my point of view, the real strength of after effects is due to the plugins developped for it. Smode have no visual plugins, everything is included with the software.
For example: Particular for AE is very advanced, But Smode allready included particles system is powerfull too, but maybe not as particular.

Without those plugins, After effects is a bit limited.
Smode has the advantage to be a native 2D/3D compositing software.
For my point of view, the real strength of the compositing inside Smode is in the masks logic. In after effects, applying a mask over an effect only mask it's rendered. In Smode, the mask play on the intensity of the effects. It's a way that offer more possibilities.

Realtime can't be ignored is the list of advantages: you don't have to wait each frame to see your result.

[illtellyoulater]

applying a mask over an effect only mask it's rendered

I have some issues understanding the above sentence; I think it has some sort of grammatical glitch which is strongly messing up my brain as I try to read it

Anyway, in general your answer was very informative! I have now a clearer picture about how the two software differ, and their advantages compared to each other.

Regarding the effect we were talking about, I am not sure whether the solution you imagined (using multiple timelines for each line of pixels) would work or not, but the fact that Smode operates in real-time certainly has the advantage of pushing me to experiment with it...

So in the end thanks a lot for your precious help @Walter_heen !
And thanks to everyone else at Smode for this awesome software!