Now that we had all the basic requirements for the filming process, scouted locations, had a storyboard and shot list, we got down to capturing the footage and breaking down each shot into what we wanted to do. To share the files we used OneDrive, so that everyone has access to uncompressed footage.
There are 3 main jobs in our plan: compositing (clean-up, rotoscoping, adding 2D elements like graffiti), portal modelling with the use of particle simulation (in Houdini or Maya) and CG world modelling in Maya for the final location.
Red – clean-up (+roto of the man), blue – portal additionRed – clean-up (+roto of people), blue – addition of graffitiSimple addition of portal and maybe particle simulation effects of dust/smoke on the floorAddition of portal and lightAddition of a different portal + rotoRed – clean-up + roto of peopleAddition of portal, roto of man, roto of hat which flies of his headCG world creation, roto of woman, roto of the arches, portal (as in location 4) creationCG world creation, roto of woman, roto of the arches, portal (as in location 4) creation
We then created the VFX schedule and assigned various shots to each member of the team, based on their personal preferences and strengths. Personally, I will be focusing on creating the portal for the graffiti tunnel, with the dust and smoke coming from the floor, as well as tracking shot 9, to export that information and to prove it to Alex for the CG world creation. I will also focus on cleaning up the harder bit (lamp) in shot 6 as well as rotoscoping myself in shot 9.
VFX schedule, weeks 5-8VFX schedule, weeks 9-10
After compiling the whole footage, I made a quick edit of the project so we all would have the correct cuts and some visual guide to see what we want to have in the end.
During this week we focused on developing the concept in a structured storyline with the script, which was then broken down into a shot list and the storyboard was created. We also scouted for locations, creating a bigger list which we then filtered down to 5 main places to be used.
Madeleine was in charge of the script, which she wrote very quickly and professionally. Going over the first version and choosing our locations, we noted down which bits would need to get changed, but overall, the script remained pretty much very similar.
Script – page 1
After the script was updated we noted down a shot list, where we decided to have roughly 9 shots in total. We also thought about the production elements such as props, costumes, potential VFX for each shot, and the sounds that we would need to get. Evanna and Alex then created a storyboard for all the locations, thus providing us with some visual clues as to how we will set up the camera and create the composition.
Storyboard by Evanna for locations 1,3,4 and 5Storyboard by Alex for location 2
During the week, Gherardo and I went to the locations to get rough shots of the places, and to note down any backup or extra location that we might need to shoot. Here the locations we will be shooting at:
Location 1 – Maggie Blake’s Cause
Arch from location 1
Location 2 – Graffiti Tunnel
Location 3 – Holborn Viaduct
Location 4 – Statue nearby Blackfriars
Location 5 – Lincoln’s Inn Chapel near Chancery Lane
Location 5 backup – Temple Church.
We then decided on which days that will be our shooting schedule keeping in mind of the weather forecast and which would suit which environment. The shooting is divided into two parts: one on the rainy day for all the places which are covered and the second on a sunnier/cloudy day where the spaces are open. I also made a rough VFX schedule plan to outline the further work that we will need to do until the deadline, which we will update once we obtain all the footage.
Following on from the last week, I continued researching how I want the second power and character to look like as well as general idea for the background, which I would either create or use an asset from online resource.
For the second character, I started off with Captain Marvel. However, I really liked the idea of a forest sorceress, having some power based around elements, specifically wind or fire-shaped power.
Windpower character moodboard
For the background at first I went into game concept art for background with fantasy or sci-fi lands, but I also envied the idea of simple black background with some neon lighting.
Choose your character moodboard
I also proceeded working further on the first power, trying to achieve the shape that I would have like the best and I got down to the following:
Aura particles
Aura particles node graph
Main power path
Main power node graph
Main particles
Particles DOP network
Houdini scene
Power tracked to hand – test 2
I am not too happy with the colors just yet and how well it is seen against the skin or lighter colors, so there is still some work that is needed to be done. But overall, I am way happier about the shape and particle formation, and path.
In the first weeks of the term, we narrowed down a couple of ideas that were interesting enough for all of us. Starting of a story about a different, undiscovered dimension inside a mirror we came to a various version of the usage of portals and finalised our first idea to be based around a few characters travelling through London via portals created from thin air or with a use of a day-to-day object, who are competing to get to the final destination. We wanted to involve a use of 3D modelling to make an animal character, like a mouse or a bee, as well as a couple of clues around the scene which will point to the direction of where the characters are going to.
Mindmap for idea AMoodboard for idea A
For the second option, Gherardo suggested using an idea about uncovering an object or a set-up made from layers. At first the room or the object appears to be one way, but as you change the perspective, it is being revealed that there is much more to it and it is consisting of various layers.
After narrowing down the previous ideas for the project, I realised that in order to get a better and more effective result I need to simplify the overall concept for the story. Instead of thinking of various shots for a fighting scene, which may require a lot of footage where hands and their movement will be tracked, rotoed and cleaned-up, I can focus on a less complicated set-up.
Having recently seen the “Choose Your Character” trend, I thought that this would be a great idea to implement. The focus would be on two characters, with a choice of their outfits, equipment and powers.
Choose your character idea
Following on from the last week I started to work on the first power which was inspired by Wanda Maximoff from Marvel universe. I managed to find this tutorial (see below), which I followed and rendered out first 120 frames of the effect. Then I set-up the scene in Nuke and using the camera tracker (3D) I got to place a card nearby the moving hand.
As I was simply following the tutorial, the way that particles are generated and interacting is not suiting my ideas. My next task would be to adjust the path and figure out of how I want them to follow around in the space, maybe of a more sphere shape. It is also quite important to understand how much that hand can move, as there is a problem of perspective and interaction with the hand.
Wanda moodboard
I am still not quite sure of how I want the second character to look and what their power will be, so my next steps will include noting down their power, simulating it in Houdini, creating a moodboard for the second character, figuring out which shots would be needed for the gameplay and what choice of style, weapon and positions the characters would be shot in.
For my personal project I want to dive into the field of superheroes/people with powers. Having a couple of ideas in my head for what it could be, it is really integral for me to break them all down and see what it more possible and can be realistically achieved in the given amount of time.
Element powers – 1 character
As my first idea it is based around having power over elements and controlling them. Based around the 4 elements: earth, air, fire and water, I was thinking of having a shot of a person manipulating the various elements and playing around with them.
Toph from Avatar: The Last Airbender – art by Aaron RodriguezArt by C. Michael LanningEarth Element powerAir element – Avatar ArtAir element – Avatar
Air tornado
Four elements – Art
Air tornado ideas
Water levitation
Katara – Avatar
Idea of water simulation
Acquaman – Mera Fire element power
Fire element
Control of fire by hand
All 4 elements in their ‘vessels’
2/3 characters
My second idea was created on a scene of two characters fighting, each having powers inspired by specific characters from action movies produced by Marvel. The first one being Captain Marvel and her power of being able to manipulate cosmic energy which is appearing as a volumetric bundle.
The second is more based around Wanda Maximoff power, where she has this red-coloured mist energy emanating from her hands and body and is able to telekinetically move objects.
Avengers: Age of Ultron – psychotic powers.Concept ArtAvengers: Infinity War – Wanda powers.
Captain America: Civil War
Concept Art
Close up shot of the powers
Malificent – similar powers
Malificent
Scarlet Witch all scenes with her powers in movies.
Scarlet Witch with all her powers in series.
The storyline would be based around the two characters, having those powers, fighting each other, at some point interrupted by a 3rd person, who plans on destroying everything, so they team up and fight against him.
Having a plate image, a couple of reference pictures with the grey sphere, chroma sphere and macbeth chart it is important to match the exposure of the reference footage to the plate. Using Nuke, we first find an object which we will be referencing to. In our case it was a wooden box.
As the file is opened up in Nuke, we note down the average value of the colour in the chosen area taken from the plate. Then the exposure is adjusted for other reference plates the following way:
1) Choose area which we used in reference 2) Create Exposure node in node graph 3) Change ‘Adjust in’ to ‘Stops’ 4) Having a reference on values for Red, Blue and Green channels from plate increase the numbers 5) Add ‘Multiply (Math)’ node in the node graph 6) For each channel adjust the value accordingly to match to the earlier noted down values.
Once completing the procedure for other reference pictures, we can move onto creating a North, South domes alongside with any extra HDRIs for too bright lights sources (in this case the fluorescent light). First, it’s important to separate the Low and High Pass to see which sources are too bright. That is done by using the ‘Clamp’ node and Merge nodes with correct operation.
Use of clamp node for Low Pass
As the High Pass is created with the separation of overly bright sources (using ‘Merge (minus)’ node with A being the source and B being source with applied clamp), the fluorescent light is separated out with a use of garbage mask, it is then fed back into the Low pass with a use of ‘Merge (plus)’ node. This way (see below), the balanced light sources are included back. That image is then separated into North and South domes.
Addition of higher exposure light areas into the low pass. Can be seen by lowering the aperture.Node graph for separation into north and south domes.
A similar procedure is done for the fluorescent light. It is also important to make a so-called ‘Sanity Check’ to make sure that none areas were affected or lost. It is done by merging the north, south domes and fluorescent light passes together and subtracting from the adjusted (in exposure) picture. If the result is black at most exposures other than extreme high numbers, then it’s all done and ready for render out to be put into Maya scene.
Sanity check when all is correct. Note the value of exposure.Sanity check when something went wrong. Note the value of exposure.
As the scene is now set up in Maya, it’s a good procedure to create separate groups for spheres, reference geometry, SkyDomes and Characters. This way we can easily maneuver the scene and hide/unhide the required groups. Firstly we create the sphere and apply correct textures on those. Then the skydomes are created, separate for each written out HDRI. Those are adjusted in orientation to match the reference plate pictures, which are also positioned in the scene so we can see then in render view.
North Dome
South dome
Fluorescent light domeRender out picture. Comparison of 3D object and the reference photos (at the bottom).Hiding the spheres, leaving table top and placing the characters in the scene.
The lighting can still be more adjusted, there are some problems such as warmer tones in the shadows, no reflection of characters in the shiny black surface on table, harsh shadows of those on the table, not enough bounce light.
Now that everyone has completed their part in preparation and creation of the assets to be inserted into the scene, the final stage was to combine everything together in Nuke. The scene ended up being quite heavy in Maya, so it took all together roughly 36 hours to render out the 1676 frames that we had in the end. Once everything was compiled together, I was in charge of compositing for the project. Jane provided the animated painting sequence, the parallax painting and the cleaned-up background shot, whilst Giulia was in charge of compiling and rendering out the scene in Maya.
Nuke node graph
I started off by writing in the passes (Beauty, Diffuse, Shadow Matte and Specular) which were then merged together and combined with the cleaned up version of the original footage. I then also added a blur effect, as there were a couple of seconds when the camera wasn’t in focus, and motion blur to match the movement. Due to some circumstances, there were missing frames for the fire render. This mistake was solved by exclusively rendering out just the fire volume with the stand for those specific frames and later merging them in Nuke. I used a planar track for the roto shape of the asset to have the perfect movement and insertion of the fire stand into the missing frames.
3D assets
Noise and motion blur
Fire missing frames
Upon merging the sequence of Maya models, I then focused on placing the paintings into the shot. For the perfect track of the frames it was intuitive to use the planar track, alongside with the key framing of corner pins of the drawing for subtle adjustments. Placing the parallax painting was the final missing piece and it took me a few tries to figure out the best way possible. At first I had an idea of planar tracking the painting to the shot, writing out the sequence and then creating cards and following the steps from our Nuke week 6 and 7 exercises. But it was quite hard to place the cards, as the room in the painting itself is small and it was tough to see correct perspective and imagine how such a picture would act in a physical space. Instead, I went through with a second way, where I rotoscoped out the walls and the floor on 4 various cards, arranged those in the best suitable way and then key framed the corner pins of those parts, to fit with correct perspective in the shot.
Parallax work
As a final sprinkle, Jane added the sound effects to give it a more submerged feel that the person is in the gallery. It guides the audience to the uniqueness of each piece, and the whole composition feels more finished.
The Outcome
The Final shotComparison at frame 0Comparison at frame 999
Self Reflection
Coming from different backgrounds and having experience in various software, it only played to our advantage as we successfully managed to split the responsibilities and give each other tips. One of such examples that I could remember of was when Giulia explained and guided Jane how to create and import the animated sequence of Keith Haring figures on the texture of the statue. Giulia was invested in modelling and compiling everything together in Maya, whilst Jane really enjoyed providing the hand-drawn paintings and animations, alongside with Nuke work on the clean up of the entire footage. As I was enjoying the recently introduced Houdini classes the most, I was really excited to create the fire simulation and adjust it accordingly given the girls reviews.
Trying to manage the workload and organization of the project, we tested various websites and online services. But in the end the most used was the WhatsApp messenger and weekly Teams meetings, in which we would discuss questions, potential ideas and how to execute them, what and whom to ask for any problems, etc. As the resource share, it was incredibly useful to have shared folder on OneDrive, as we would drop everything there. The schedule was kept in check and when needed we would agree on excluding anything that we might have wanted to do originally, but couldn’t execute due to the lack of time.
Judging my own work, I was pleased with how the fire turned out, however it was a shame that I didn’t manage to finish the moving particles around the statue base, as originally thought of. Starting out the script in Houdini and research I managed to get to a point of having some particles moving around an object, but work had yet to be done to get the better look. Due to the time restrictions it was decided to leave it and focus on the rest of the project. With the compositing, again, the final result was satisfying, but there were a few mistakes that I only saw after writing the sequence out. One of them would have been creating the roto for the first and last few frames to make a gradient for the planes created in Maya, served as shadow mattes, which have outlines that are seen.
Houdini node graph
Part 1
Part 2
Houdini viewport scene
Overall, I really enjoyed this project and am very happy with the outcome. Having not achieved everything that we wanted, it was still a great experience to go through with it, understand teamwork, communication and time management. From the beginning our group was very well balanced, we agreed upon a lot of ideas and provided great support and help to each other. It was quite easy to get a constructive criticism and we would always agree on any changes required.
After showing Nick the render from previous week, it was pointed out to me that the lighting was incorrect. I adjusted the lighting (see below), to have more defined shadows of the machine appearing on the backwall and the left side wall. Due to the lack of time, I only rendered out the new shadow pass, such that the the new shadows would get combined with the beauty pass of previously lit up scene. Here is the result.
First render
Second render
Final shot
I also rendered out a frame of the new beauty pass, to see how it would have looked in the scene with the new lighting:
Different lighting render
When the rendered files were imported inside Nuke, I didn’t break them down into different passes and combined back together, as the final output would look different to the beauty pass. The only exception was for the shadow layer, as the beauty and shadow render layers were exported separately, thus giving more control over the 3D asset.
I did use the ID passes, to affect the colour of the wheel, make it slightly deeper and darker tone. Same was applied to the golden features of the machine on top of it. An attention had to be payed specifically for couple of objects that appear to be in front of the machine. For those, I created a roto shape, which I then planar tracked and combined together using the merge node with operation ‘over’. Inside the lamp, however, is supposed to be some glass object, maybe see-through space, so a roto for it had to be cut out from the previously combined objects. All these alternations then were used as an alpha channel for the mask, to put original footage over the merged.
Lamp roto
Bricks roto
Rock roto
Inside of lamp roto
Alpha channel
Nuke node graph
Self Reflection
Looking back at the entire process I have learnt a lot about the integration of modelling and compositing in such an extensive project. I learned more about lighting, animation, rendering in itself, how to manipulate various passes and adjust potential mistakes later in Nuke. There were a few things which I was proud of, but not without a couple of mistakes and few realizations of what and how it could be done better.
Firstly, I could have implemented various machine parts, without attempting to copy entirely a whole machine from the original source. If I had done more of research and understanding of where it would have been placed, it would have been more clear to me of whether having a lot of small details was necessary. After all, the machine model itself has a couple of detailed bits which are barely seen or not at all. Secondly, I would have wanted to implement more of rigging, rather than just animating parts. Having encountered problem of understanding how to rig specific parts, I ended up using constraints, joints and IK only on the curve-shaped rod attached to the rotating wheel.
After a couple of weeks, modelling of non-organic objects became easier and easier, so I really enjoyed modelling the smaller bits or unusually shaped parts. When unwrapping the UVs it took me a couple of tries to understand how that would be working. As most of the parts weren’t seen much or actively animated, I attended the parts that needed most attention. Texturing was fairly easy and straightforward, especially with an outside look: pointing out what is unrealistic and how to make it look more worn down. Using Mudbox to make my own textures for the wheel and screws gave me some room for creativity and unique attention to those parts.
At the final stages of lighting the scene and trying various renders, we found that the HDRI was from the first room and thus created incorrect lighting for the scene inside the second room. Via trial and error, I placed lighting around the scene and experimented with the settings of exposure and intensity, to match the lighting reference of the rocks on the ground. I rendered out the variations of the scene and placed into Nuke script, to see how it would look in the end. The biggest mistake that I made in my final render was the use of HDRI to cast shadows and not blocking out enough light from it. The light coming from the left side of the room would create the highlights and specular details on the left side of the machine (e.g. edge of the wheel), which don’t match the shadows that are stretched out towards that direction. As I adjusted the lights after constructive criticism from Nick, I rendered out a single frame to see how the lighting would work for the whole scene, whilst rendering out the entire sequence just for the shadows. Overall, the shadows do look better, but they contrast slightly with the specular details which don’t belong there.
All things considered, I would re-organize my own time and shift my focus on breaking project down into a more detailed plan, such that I could see overall picture and leave reasonable time for render tests. Despite all the mistakes and issues along the way, I am pleased and satisfied with the result. This project gave me some ideas of what I want to do further in the software and what I want to learn about more.
