Towards the final term, I have decided that I wanted to focus on FX simulations and hence I started working on ideas of which type of simulations exactly I wanted to try. From my long-time idea about interaction with all 4 elements to focusing on a specific type of element, I landed on wanting to experiment with liquid simulations. Working around some kind of storyline, I wanted to have two main plot ideas of relaxation and then shock, in a contrast to each other. Within each emotion, the FX simulation would have been the key element.
Setting the mood of an office environment, my story was based around a worker who decided to take a break and make themselves a much-deserved cup of coffee. The process of preparing ingredients and combining them all together would have been the moment of relaxation: with a culmination when they added milk, thus having a shot seeing how the milk would fall into the darker liquid of coffee and mix together. That would be the first main moment of liquid simulation, specifically a combination of two and conveying precisely how the liquids would interact with each other. Followed by a visual supportive action of the worker getting a few seconds of happiness and relief, their next step would be going back to their space and sitting down in the chair. As they would have sat down, the hand with the coffee would start placing the cup on the table but miss out the edge of it by a few centimeters, thus misplacing the cup, The following few shots would have been a slowed-down version of the cup falling and its content spilling all out, maybe taking a shape of something mid-air. Finally, the cup would fall and break, speeding up in time and ending the story. The last actions would be simulated as well, both the unusual shots of the falling cup with moving liquid and its crash.
Storyboard, 1-12Storyboard, 13-22
Liquid morphing into a shape reference.
Liquid taking shapes due to external force, such as sound waves – reference.
Liquid mixing up together – CG reference.
Ink mixing ideas – reference.
Milk mixing with tea – real life liquid mixture reference.
After creating the storyboard, I made animatics and filmed the first version of test shots, to see the flow of the story, how it would develop, and if it would show correct emphasis on my VFX work. The peer-review helped me to understand that I had too many shots prior to the first fluid interaction shot and that I would need to refine the story. It was also pointed out to give more thoughts to what kind of shape I wanted the fluid to take, maybe it would be something personal to the worker, a state of their troubled mind or worries.
Having received the feedback, I now knew what my next steps would be and which parts of the story and background I need to give more consideration to.
This was one of the toughest projects to work on during this whole term. Being appointed as project manager coordinator by Aldo, it became my responsibility to organise meetings and check up on all teammates to make sure they know and are happy with what their tasks were. Using Microsoft Teams was very convenient to schedule and have video meetings, whilst OneDrive became an excellent resource where we shared our files, Nuke scripts, and renders.
Starting of the brainstorm, we narrowed down the ideas to the main being about traveling through London with the use of various portals and the second about layers on top and revealing the layered scene by changing perspective. With more research, we confirmed the first choice and started working on scouting the locations, planning shooting days, and thinking about the shots. With the help of her director’s background, Madeleine made the script, which helped us to visualize the shots better. Yingyi and Xuran tried themselves out as storyboard artists, both of which looked great. Finally, we planned the shots, wrote down the Shot List, and started filming. It took us 2 days and most of the team was on the set, which was quite convenient.
After we got all the shots, now was the time to plan out the VFX schedule. I dropped down the first edit, based on everyone’s preference and strengths. During the meeting we perfected the plan and spread out the jobs, but not quite evenly, as some teammates were complaining about specific software not working for them. After that everyone got down to working on their shots, and we had weekly meetings to see where everyone was at as well as to help out or give suggestions or ideas on how to solve various problems.
My tasks included creating a portal for scene 2, shots 3 and 4, a portal for scene 4, shot 7, and for scene 5, shots 8 and 9. I was also in charge of compositing shots 3 and 4, had to help out Madeleine with clean-up in shot 6 (cleaning the lamp, precisely), and, as of last week, of compositing shot 9. It was a lot of work, which I couldn’t do in time at 100% each, so in some of the composites, I am not as happy with the results. For instance, the portal in shots 3 and 4 still stands out a bit too much and not as integrated into the scene. The light that it emits onto the surroundings still seems quite fake and not as realistic. Maybe using a light source with the correct colour on the day of filming would have solved that problem. I also tried manually making the smoke and dust to be flying off the floor, as the portal would open up and is a source of energy. A good suggestion from Mehdi was to use a simple shape of an increasing circle to create that burst of energy to move the particles and the smoke, however, I couldn’t get any good results in time. For shot 4, due to the perspective, it was easier to submerge the character inside of it and hence fake them exiting. To make it further realistic, probably a 3D model of the character had to be created and inserted into Houdini to simulate that wind and dust interaction as they exit from the portal. For portal in scene 4 (which the male character was jumping through), it was decided to go with the image of fire. The thing it lacked was more details and hence more realism. But it would mean that the voxels inside Houdini would be very small, and my computer was crashing every time I tried to adjust it and make them look smaller. Most likely, there is a way to solve it, and it’s one of the things I want to attempt again, after submission of the project. Shot 7 was one of the shots that technically would have required more thought prior to filming and most likely extra props during the filming. In shot 9, as I made a couple of unsuccessful attempts of creating the newly looking portal, I was running out of time and decided to use the first version of the portal I made for the easier shots 3 and 4. To make it look distinct, I changed the shader to be of green colour for this shot. Compositing of shot 9 was quite tough, we had to deal with overexposure from background light, causing faces, hair or back of some characters to be overlit and showing white blobs of light. With advanced planning of dividing the rotoscope between Madeleine and me, it made the workload slightly less, so I could focus on fixings other problems, like the position of CG assets, their movement, grading the portal to fit the shot, adding light effects like volume or god rays. Overall, that was the toughest shot for me, but I’ve learnt a lot. One of the biggest lessons taken from this project would have been thinking about lighting in advance, analysing the location, using green screen parts where possible such that we could have a better interaction with future CG assets.
As of the final week, due to the fact that no one replied or said that they didn’t have the time when it was asked for whom to be the editor, I volunteered to take on that role. That meant, that I would have to download everyone’s latest versions of the shot and work on sound effect edit, suggest few music tracks for which the team would vote, color grade it, and add the credits or any text. With a few miscommunications about who will be compositing the shot 8, the last week became harder to manage, both as the manager coordinator, as well as coping with the personal project and the Indie Horror Movie project. I started to regret a few of the decisions and choices that I made. The hardest part, which still affected me negatively a lot, was getting into an argument with one of the teammates, who decided that it was their decision to make changes to compositions of shots 8 and 9 a day before the deadline, without prior conversation and suggestions with the team. What I found absolutely disrespectful from their side was that they messaged one of the tutors the night before the deadline just to get a supportive and permissive opinion from a more authoritative figure, whilst still not saying a word about it to the team, only to use it on the day of the deadline as a strong argument as to why we should be using those shots. The actions of that person do not correspond to any definition of the words “collaboration” and “teamwork”. Most likely, some teammates will have a different version of the edit of the final project.
Overall, I am satisfied just enough with the final outcome of this project. The problem was a directly incorrect division of the tasks, evaluating what people are actually capable of, and being more strict to some of them. Had I myself had less work to do, I would have managed to produce better results. Still, this was an experience, as after all, one can never know what kind of people they will get as their teammates and a VFX artist not only has to produce good work, but also be a good team worker, kind, supportive, and just a nice person with whom the co-workers can get along.
Finally, I worked on the addition of few animations made in After Effects, a soundtrack, and sound effects to get more of a feel being submerged into a real video game character choice menu. It was hard to choose, but I decided to go with simpler but more cartoony effects, instead of realistic-looking ones. Personally, After Effects is not the software I am too confident with, so in a few days’ time, it seemed like a logical and reasonable choice to go with such animations. I used the following tutorials: (https://www.youtube.com/watch?v=w8mHmW6GgHM) for the smoke bit, and (https://www.youtube.com/watch?v=F-dfXzn90sc) for electricity bit.
Final Outcome
Final Project
Shots breakdown
Personal Reel
Self Reflection
In the beginning it was tough to narrow down the options and figuring out what was it exactly that I wanted to do. Yet it was a great starting point, when I knew that I wanted to go down the path to do with effect creation in Houdini and compositing the shot in Nuke. After some research and dropping down a couple of ideas, I settled with the one of creating a sequence of two characters fighting each other. However, after weighing out the work I had to complete for my collaboration project, as well as the Indie Horror film, I changed the concept to be based around choosing a character, where each would demonstrate in some way their power.
One of the great ideas was to shoot my own footage and test out the particle renders to check the tracking, how it would look against backgrounds with various colours, whether the particle simulation would required hand animation inside Houdini or not. Finalizing the look of the first power didn’t take long, surprisingly, although now that I look at it, it feels like it misses more manipulation and integration with the hand. After having the first power done, I moved onto the second, which turned out to be the harder target for me. Having been inspired by footage from Malificent (2014), I wanted the energy to create vortex shapes that would follow the path of the fingers, as the actress would change their position and play around with the power. Yet again, I had to step back and restrict myself to a power that would be more possible to achieve in the constantly reducing amount of time. So instead I focused on combining both the wind render, which gave it a more chaotic look alongside with directed particles swirling in the upward direction.
The next step after research and tests was to note down the shot list, prepare a rough edit for reference to me as a videographer, and for the actresses to take inspiration from and set the filming date. Being pushed back a couple of times, it was a nice coincidence that the shooting days were planned after we had an introduction to the green screen room and taught by Aldo of all the necessary points we have to think of in advance. Those included making sure that the background was well and evenly lit, how to use the lightmeter, adjusting the lights as it was in the background sequence. Since mine was an imaginary place, I had more freedom of how to light my characters, so that took away hard bits. However, with some of the props I didn’t realise that they had green or turquoise undertones to them, so when removing the green screen that affected the colour of the objects and one of the shots was impossible to use.
As soon as I had the shots, I immediately narrowed it down by editing first to see which would suit the best for the final product and to reduce the number of shots that required green screen removal. From 16 original shots, there were now 8 to compose and adjust, for which I tried creating the one-for-all Nuke script in order to remove the green screen. It managed to cut down the amount of time spent on working in Nuke, whilst requiring only small adjustments, like rough roto from which the shot was premultiplied to put against the new background or roto for the hair, to get a more detailed alpha channel. A mistake that I’ve made during filming, was directing not enough light on the characters, meaning that the textures of the dark clothes weren’t captured, and later when I applied the LightWrap effect, it appeared quite flat in the shot. I’ve also experimented with adjusting the power look inside Houdini and how it would affect the hand holding it.
Finally, compiling all the edited shots together it was now easier to see mistakes and get feedback from the tutors. Nick suggested adding reflection to the floor, to base the characters better in the environment as well as implementation of some effects to when the characters have weapons, but no particle simulation. This was very helpful advice, so during the last week and a half I focused on creating the according adjustments and inserting sound effects for the edit.
Overall, I enjoyed this process a lot! It was hard, intense, yet interesting to further learn Houdini and discover new ways of creating magic effects within the software. Some decisions required cutting down original grand ideas and analysing what would be more possible to do in given amount of time, yet would still push me. Had I said no to the Indie Horror project and have been strict when dividing the work more evenly for the collaboration project, I would have had enough time to experiment more and maybe achieve a better looking results from Houdini. This term helped me a lot to learn about my strengths and weaknesses and pushed me to ask for help and feedback more, which is something I am not very used to.
This week I proceeded to update and finalize the compositing of shots 3 and 4. As suggested I added distortion to space just before the portal appears mid-air. I used the Noise nodes, one of the red channel, the other of the green channel, and adjusted the size of the noise to be combined into the original footage and be used to distort the image. With the use of a roto, I merged it with the Original Plate for which the operation was Difference, to get as close as possible to resemble visual tearing in space.
Part 1 of Nuke scriptDistortion of the space with use of masking and Operation DifferencePart 2 of Nuke Script
Distorted footage for the portal
Part 3 of Nuke Script – dust particle addition
After feedback from my teammates and Aldo, I also added grade to the space and extra Volume Rays to give the feel like the portal is emitting the blue light and hence affecting the colours of the surroundings. I’ve also added a reflection on the floor of the portal to help it settle in the space better.
For shot 4 I’ve mainly adjusted the roto of the character, tried to make a cleaner integration with the portal, added colder grade and volume rays to space. I also rotoed out the shadow coming from the character and added it back after merging portal shadow with the original plate. Now that I look back at this, it isn’t necessary, but in an earlier version, I was placing a roto of the clean floor back onto the footage of where the character left the portal. However, it was messing up with the script and wasn’t even necessary as it couldn’t be seen behind the portal itself.
Shot 4 – Part 1 of Nuke Script
I also used 3D camera tracking in order to create a card for the location inside the portal and place it in the scene more correctly.
Shot 4 – Part 2 of Nuke ScriptShot 4 – Part 3 of Nuke ScriptShot 4 – Part 1 of Nuke Script
For the portal in shot 7, through which the second character was jumping, I got the feedback asking to make it more detailed and realistic on which I worked in Houdini set-up. For more realism, it was essential to lower the size of the voxel for the fire, the division of the bounding box in which the smoke for the fire was generated, and apply a better shader. If for the other portals I used the shaders built-in and Mantra renderer, for this portal I decided to set up the Arnold Renderer in Houdini. It required an extra node in the obj menu for the volume to be visualized in the renderer, as well as a specific Arnold Shader for the fire.
Obj menu set-up for Arnold renderArnold material for the fireFire render
For shots 8 and 9 to save time it was decided to implement an earlier version of the portal in scene 2. To make it look different, I applied a green colour in the shader.
Render of the portal against the background
It was my job to composite the shot 9, for which Xuran created the CG background, that we see inside the arches. To divide the work among us, Madeleine rotoscoped out one of the characters in that shot, and it was my job to rotoscope the second character. To save time, I didn’t rotoscope all the parts, but the ones that would appear in front of the arch.
Nuke – whole script
One of the hardest parts was to fix the tracking of the CG assets, as they were manually tracked along with the picture background. With the use of transform node, I key-tracked the movement separately for the background, as the image on it is supposed to be miles away from the temple position; separately keyed the temple door and levitating parts on the right-hand side as when I was adjusting the door, the rendered picture of the right objects was cut off and looked incorrect (see below).
Original fixed CG render to the shot render.
Node graph – part 1
Node graph – part 2
Node graph – part 3
Node graph – part 4
The final step left is to edit the shots, add a grade, soundtracks, sound effects, and some text.
Having a good base for my project I then focused on adjusting the details and perfecting the little bits. Asking around for review and suggestions, it was also decided to change the font of the menu and add simple, yet effective animations to the punching actions later in After Effects. I also added a reflection to each green shot and a subtle glow behind each character to bring them forward visually from the background.
Adjustments for the 2nd character: 1) added glow, 2) smaller, 3) reflection on the floorChange in edit: 1) Font, 2) Glow from behind the character, 3) adjusted the green screen despill, 4) wind power look, 5) removed shadow as the power is the source of light, cannot have shadow of itself so close to the centre
Looking at the power, I didn’t like the original motion at a few couples of seconds, so it took a couple of re-adjustments to get closer to the final look I had originally envisioned. Inside the Nuke script, I’ve also added the glow to bring the character visually forward. The skin closer to the edges had more green colors in it, so it was also color graded to add slightly more purple/pink tones, in order to cancel out the green shades.
BTS: Wind power creation
Gas Geo network
Gas DOP network
Gas node graphs
Part 1
Part 2
Part 3
Particles node graph zoom inPower adjustment – comparison of test 1 and test 2
As planned I manually created the new introduction using After Effects having that glitch effect appearance with the use of this tutorial (https://www.youtube.com/watch?v=3Rc0dNDzbuM). The key to it was to animate the glitch appearance via a change in brightness of noise applied as a layer onto the text. To visually guide which character is chosen, the simulation of pressing the menu choice was added with the use of keying the change in opacity.
New introduction and menuAfter Effects Composition for Title appearance.
The final main parts left for the project are to add sound effects and background music as well as some artistic comics-style effects (made in After Effects) onto the punching actions of the characters when they have a weapon.
After my initial tests with close-up shots for hands and powers, I noted down a general plan of what kind of shots I needed. It was decided to have 2 characters (as planned) each to be filmed in casual and Ready To Fight outfits with a choice of weapons from a tote bag, water bottle, or a book. Before filming the actors in the Green Screen room, I filmed myself and made a very simplistic edit.
First edit in Premiere ProStanding shot for Character 1 – Green Screen RemovalStanding shot for Character 2 – Green Screen RemovalNuke script for green screen removal
Green Screen tone adjustment
Alpha pipe
Despill pipe
Merge of despill and roto
I then made a rough cut for the edit of how I want to see the final video. Finally settling on the background it was easier to remove the green screen and start creating the clip. For the finalized version, I still want to work more on the text, font, and how it will be stylized. This week the plan is to also work on the title appearance, the end of the clip, as well as power adjustment against the green screen.
Wind power first render test
I have found wind power to be harder to test and adjust as it has its own nuances and problems. For instance, it needs tracking and more trials for the look in Houdini, before I could render out any version.
Wing hand clean-up
Wind GS removal
Wind power insertion
For the next week, my focus will be around getting the first render for the wind power, implementing it into the edit, animating the title appearance, adjusting the energy power Nuke script for a more realistic result, and researching for a way to make the zoom-in effect more sleek and correct. I also want to start searching for a good background soundtrack and some sound effects to accompany the edit.
Over the next weeks, I focused on my tasks, such as the creation of Portals for Scenes 2,4 and 5, comping one of them into the shots for scene 2, and lamp clean-up from shot 6.
I started off with the hardest one, specifically for shots 3 and 4. Firstly I just experimented in Houdini and got the following results:
Test 1Test 2 – adjusted the sizeFlipbook
However I wasn’t happy with the overall movement and I wanted for it to have more control and flow from inside to the outside whilst being attracted to a central shape inside. So, after finding a great tutorial (https://www.youtube.com/watch?v=SnGSCO0CwAc) I re-made it into a different shape. I also added a light source, trying to imitate the lighting in the scene and created a rough geometry to represent the wall, floor and the sidewalk, such that the portal and smoke coming from it could interact with these.
Test 3Flipbook
Test 4 – flipbook for shots 3 and 4
Portal node graph
POP network
DOP network
Portal Houdini object node networks
Viewport
Geometry set-up node graph
Geometry for interaction set-up
I then made a simple set-up for compositing in Nuke of the rendered out versions of portals to get feedback from the group.
Nuke node graph for the Scene 2, Shot 3Nuke node graph for the Scene 2, Shot 4
I also added an image of a different location to be seen through the portal and started to figure out how to add distortion or warp effect to that image. After a couple trials including creating a Heat Wave effect with the use of nodes ‘Noise’ and ‘IDistort’, I realized that having a sequence of popular locations would be a better choice to see a more natural look. For further work on compositing the shot and achieving the final vision, I need to implement some ambient lighting, dust flying around, and put some distortion to space before the portal appears.
For the division of the workload I was assigned to a partial clean-up in shot 6, of the lamp to be exact. Proceeding with simple rotoscope, rotopaint and subtle adjustments with the use of Blur, EdgeBlur or Erode I cleaned out the lamp fairly quickly.
Shot 6 lamp clean-up
Shot 6 Nuke script
I then proceeded to make the second portal, which we decided to be of the fire shape. As a starting point, I used the Houdini script I made for the first trial of the previous portal. I then adjusted for it to be on the floor, to follow the shape of a circle from the shot and animated its appearance. The next step was to create particles and figure out the way they would be simulated in a POP network. After the look was finalized any particles below the floor level were excluded from simulation and volume was create with attributes of density, temperature, and velocity. Finally, the smoke was solved with the use of PyroSolver inside the ‘DOP network’ and later read into the main script with the use of ‘DOP Import Fields’ and converted into the VDBs. Furthermore, I added material to the fire and created a Mantra Renderer.
Flipbook of the portal in shot 7
Fire Portal Houdini Network
POP Network
DOP Network
Fire Portal Houdini script
After feedback from the group, I will adjust the portal’s animation if needed, set up the lights, set-up floor geometry for shadows and change the renderer to be Arnold and set up Arnold materials for the objects. It will also be important to find the precise number of frames such that I could animate it correctly.
Continuing on from the previous weeks, I proceed further with testing out the first power, now against a dark background and slower, more controlled movement of the hand. I also added a slight glow on the hand, using roto as a mask for colour correction. Additionally, I played around with the colour of the power, the god rays and the overall grade of the shot.
Nuke node graph
Furthermore, I tested out the first power being placed between two hands, and whilst with one hand a simple 2D tracker was suitable, this time I did have to use a 3D tracker and to try and find the best position for cards in the 3D space. The final result still appears to be attached more to the base of hand situated furthest away, so maybe by manual adjustments of the card it would be possible to make a more realist effect. I am also considering to adjust the camera in Houdini, such that the render of the power will also have the effect of changing perspective.
Two hands – energy ballNuke node graph for 2 hands
Then I started working on the wind based power, but I am not happy with the overall shape of how the particles are flowing around. Since it is now more abound wind, I am considering whether to leave the attribute of temperature out and just have it appear as smoke, rather than having a hot emitter in the centre. In that case, the God Rays node isn’t suitable anymore.
Wind power first test
Velocity guide – Houdini node graph
This is the original shape that I wanted to go for, having this sort of vortex/tornado pathway of wind, but it appeared too intense and unnatural. So playing around and creating a different source for the smoke (see below), I decided to use the information for velocity from the vortex and feed it into the source as a force for the smoke to be affected by.
Wind source – Houdini node graphDOP network for the wind power simulation
This is how the Dynamic Operator network looked with the simple force of the velocity and smoke emitter being merge and solved using a pyrosolver. Having played around with it, I considered creating a second smoke object, where instead of being directed up and by the vertex, it would fall down and wrap around the hand. But that may be way harder to do, as it would required hand modelling and animation inside Houdini. Instead, I will try and achieve a different look, as vortices going up, i.e. just as I wanted originally.
From the last lesson, we proceeded to create the destruction to the other parts of the building, such as walls, bricks, supporting bits, and glass. For a faster way, the use of ‘For’ loops was implemented in the workflow, thus applying similar procedural destruction and noise to a lot of smaller pieces.
Bigger concrete destruction with meteor area selected (Active_zone)
‘For’ loop for the concrete destruction pattern
Glass destruction
To make the process more optimized, as the meteor would only interact with specific parts of the building, we used the shape of a circle following a path of the meteor through the building (over about 6 frames) to select the surface that would have the interactions and apply more detailed patterns of destruction only to that area. Then all the various bits would get combined back into the building and proxy geometry would get created for the bullet solver to be able to handle the operations.
All destruction file cache merged and proxy geometry created
After finalizing where the lines of breaking points would be, we then added the constraints to guide how it will break and how strong the force of the glue holding it together will be with the use of “RBD Constraints from Rules” followed by “RBD Constraints Properties”. Then the building was divided into active and non-active sections and combined together with the bullet solver, which read the information for collision geometry of the Meteor Proxy that we made earlier. Hence we finally got a look at how the building will get destroyed once in contact with the meteor.
RBD and Bullet solver for active geometryMeteor colliding with the building
As there were two main contact points of destruction, to add realism we separated them out and made explosions at each point of entrance/exit. First, the volume geo was made by using the meteor proxy taken from a single frame, visualized with the “Trail Path” node, and changed into the volume. For every explosion, the trail path was adjusted to see visually how we wanted the particles to move.
2 points of contact Meteor Volume Geo
Example of trail path
Once the trail paths looked good, they were combined with the Collision geometry, and with the use of “Pyro Solver”, we achieved the final look of the explosion for each point of interaction.
Pyro solving with the use of Volume and Collision Geo coming from Meteor proxy and Active group for destruction.Destruction with explosions
First explosion: pyro solver settings
Second explosion: pyro solver settings
The final addition was the smoke/dust coming from the separated parts of the building, resembling the dust from smaller particles of concrete broken apart. I managed to follow the tutorial and write out the first 15 frames for the required simulation, but my computer would crash after the frame 38 and not write out any caches, so my final video is of the destruction and smoke by themselves.
Explosion from broken pieces
Smoke overall lookDestruction with the final smoke
Overall, these two weeks of lessons were quite intense requiring a lot of computer power and memory, which was an obstacle in my case. I really enjoyed breaking down the process and messing around with details of instruments to get my own view of the destruction. However, looking back at the result it feels like the pieces at the second collision (a lower bit of the building) have too much force applied and are flying out too far out, which wouldn’t be the case in real-world as the meteor would have slightly less force at the 2nd point of exit.
