Category: Collaboration project

The Final Stage

Now that everyone has completed their part in preparation and creation of the assets to be inserted into the scene, the final stage was to combine everything together in Nuke. The scene ended up being quite heavy in Maya, so it took all together roughly 36 hours to render out the 1676 frames that we had in the end. Once everything was compiled together, I was in charge of compositing for the project. Jane provided the animated painting sequence, the parallax painting and the cleaned-up background shot, whilst Giulia was in charge of compiling and rendering out the scene in Maya.

I started off by writing in the passes (Beauty, Diffuse, Shadow Matte and Specular) which were then merged together and combined with the cleaned up version of the original footage. I then also added a blur effect, as there were a couple of seconds when the camera wasn’t in focus, and motion blur to match the movement. Due to some circumstances, there were missing frames for the fire render. This mistake was solved by exclusively rendering out just the fire volume with the stand for those specific frames and later merging them in Nuke. I used a planar track for the roto shape of the asset to have the perfect movement and insertion of the fire stand into the missing frames.

• 

• 

• 

Upon merging the sequence of Maya models, I then focused on placing the paintings into the shot. For the perfect track of the frames it was intuitive to use the planar track, alongside with the key framing of corner pins of the drawing for subtle adjustments. Placing the parallax painting was the final missing piece and it took me a few tries to figure out the best way possible. At first I had an idea of planar tracking the painting to the shot, writing out the sequence and then creating cards and following the steps from our Nuke week 6 and 7 exercises. But it was quite hard to place the cards, as the room in the painting itself is small and it was tough to see correct perspective and imagine how such a picture would act in a physical space. Instead, I went through with a second way, where I rotoscoped out the walls and the floor on 4 various cards, arranged those in the best suitable way and then key framed the corner pins of those parts, to fit with correct perspective in the shot.

• 

• 

• 

• 

As a final sprinkle, Jane added the sound effects to give it a more submerged feel that the person is in the gallery. It guides the audience to the uniqueness of each piece, and the whole composition feels more finished.

The Outcome

Self Reflection

Coming from different backgrounds and having experience in various software, it only played to our advantage as we successfully managed to split the responsibilities and give each other tips. One of such examples that I could remember of was when Giulia explained and guided Jane how to create and import the animated sequence of Keith Haring figures on the texture of the statue. Giulia was invested in modelling and compiling everything together in Maya, whilst Jane really enjoyed providing the hand-drawn paintings and animations, alongside with Nuke work on the clean up of the entire footage. As I was enjoying the recently introduced Houdini classes the most, I was really excited to create the fire simulation and adjust it accordingly given the girls reviews.

Trying to manage the workload and organization of the project, we tested various websites and online services. But in the end the most used was the WhatsApp messenger and weekly Teams meetings, in which we would discuss questions, potential ideas and how to execute them, what and whom to ask for any problems, etc. As the resource share, it was incredibly useful to have shared folder on OneDrive, as we would drop everything there. The schedule was kept in check and when needed we would agree on excluding anything that we might have wanted to do originally, but couldn’t execute due to the lack of time.

Judging my own work, I was pleased with how the fire turned out, however it was a shame that I didn’t manage to finish the moving particles around the statue base, as originally thought of. Starting out the script in Houdini and research I managed to get to a point of having some particles moving around an object, but work had yet to be done to get the better look. Due to the time restrictions it was decided to leave it and focus on the rest of the project. With the compositing, again, the final result was satisfying, but there were a few mistakes that I only saw after writing the sequence out. One of them would have been creating the roto for the first and last few frames to make a gradient for the planes created in Maya, served as shadow mattes, which have outlines that are seen.

• 

• 

• 

• 

Overall, I really enjoyed this project and am very happy with the outcome. Having not achieved everything that we wanted, it was still a great experience to go through with it, understand teamwork, communication and time management. From the beginning our group was very well balanced, we agreed upon a lot of ideas and provided great support and help to each other. It was quite easy to get a constructive criticism and we would always agree on any changes required.

During the first week of March we focused on finishing the tracking in 3DEqualiser and setting up the scene in Maya. The three of us all attended another few sessions with Dom, where we all tried perfecting the track of the shot. After looking at the footage followed by an advice from Dom, we decided to cut down the shot by a thousand frames and have it last for 44 seconds, rather than 68 seconds. This way the attention of the audience would not get lost and the shot would be more interesting.

The tracking information was then exported into Maya, after which we placed cards to represent the wall and the floor, as well as few cubes to represent the black stands. We also added lights, position of which was based on the light pattern falling on walls and the floor. Test objects such as cube and few copies of a torus were placed around the scene for a render run.

Earlier, during the project planning stage, Jane found an Egyptian statue. When we met last week, we decided to add sequences of dancing characters (Keith Haring inspired) to the texture of the statue. Proceeding with the task, Jane created a png sequence of those with the transparent background. She also re-did the animated painting of the her earliest work with the newer characters.

• 

• 

• 

• 

• 

• 

As the clip was cut down we had to adjust the plan for object placement and question if all the original pieces should be placed in the scene. Since we all equally liked the Egyptian statue, it was decided to keep it but we did cut down the amount of pictures from 3 to 2. The first one to be the painted version (by Jane) of the Van Gogh famous painting “Bedroom in Arles”, whilst the second to be the animated painting. The classical statue was kept in the same place and the display case with the fire to be placed between the two paintings.

• 

• 

• 

• 

Giulia was the one who collected most of the free assets for plants, as well as modelled the pyramid shaped display case. Furthermore, she put herself forward to model the neon signs which would be positioned around the classical statue by the black stands. Various tests for quality of neon lights were carried out by her, which we all later discussed during our recent call and came to conclusion of what look we all preferred.

• 

• 

• 

• 

• 

• 

• 

• 

For the particle work, I organized a session with Mehdi, who guided and helped me to understand how to use the object as boundary for the fire simulation. After achieving the fire look that I wanted, I also played around with material and colours of it, to found what we liked the most.

• 

• 

• 

• 

• 

• 

One of the difficulties that I came across was how to read the VDB files inside Maya and later how to export the volume material from Houdini and read that in Maya. Still in the process of finding the solution, my next steps will be trying out the methods I found in various forums, contacting Mehdi and, if all fails, exporting the render from Houdini rather than Maya. For that, I would need to export the tracked information again from 3DEqualiser to Houdini, export the objects and lights that were set in Maya to Houdini, such that the scene set up would be exact as in Maya. The display case would also be needed to get imported to Houdini and be rendered there as well, as the light is interacting with the glass properties of the case.

Other than figuring out the proper way to render the fire, my next steps involve finding all the references for particle movement around the Classical statue stand and setting up the node network in Houdini. For those, I believe, placement of material would be easier, so that could be rendered inside Maya.

The main focus of the last two weeks was on getting the footage and organizing sessions with Dom in order to track it in 3DEqualiser. With variety of shots got by Christos, who followed the latest camera previs clip, both captured on Blackmagic Pocket Cinema Camera 4K and Canon 6D, the three of us agreed on using the very first shot done of the latter camera. The crop factor on BMPC4K was quite high, x1.9, meaning it was cutting out a lot information which was required for tracking.

We then managed to get a learning session with Dom, in order to utilize 3DEqualiser for a better tracking on Monday last week. During the session we were practicing on the first 2000 frames, meaning there are about 1000-1500 more that should get tracked and have the information exported from. Another session was planned, where Dom would show us how to export all the material with tracks into Maya and how we would placed objects around.

At the meantime, Jane managed to test the parallax effects in Nuke on the footage and would send us her trials, such that we could decide on the changes where necessary.

During our team meeting on Friday, we figured out and noted down the main steps of the project workflow and how would we divide the workload between ourselves. As a conclusion this was the rough outline of what had to be done:

1. The track needs to be finished up and perfected in 3DEqualiser.
2. The footage has to be cleaned-up.
3. The objects with burning fire inside the display case and with the particle movement around the stands underneath statues needed to be completed in Houdini.
4. The 3D objects need to be placed into the scene in Maya, with the planes representing walls, other cubes to represent the black stands, lighting and maybe an additional wall behind the black stand. Their AOVs, movement and placement in the scene would then be rendered out (without the image sequence), undistorted, as a jpeg (or similar) sequence, such that that footage could be merged with cleaned-up plate in Nuke and AOVs could be adjusted accordingly where we want them to be done so.
5. The parallax paintings would be placed as cards in Nuke with the tracked information exported from 3DEqualiser.
6. We would then also apply colour grading to give the shot a more cinematic/artistic view.

The final outline for all objects placed and their purpose in the scene was the following: the first and second paintings having a parallax effect, where Jane would originally draw those paintings based on some pictures; the first classical statue would be done in the Vaporwave artistic style, for which Giulia would model neon signs and lights and place them accordingly; the 3rd painting to have some animation inside of it; the Egyptian statue to have animated texture (both done by Jane) and have a few 3D projections placed over it; the display case to be modelled by Giulia and the fire added by me in Houdini, as well as the particle movement to the stands for statues. It was also decided for the tracking to be attempted by all three of us and the best track to be chosen and used for the project.

When discussing of what is needed to be done to the statues and what would be in the paintings, we had an idea for the 3rd painting to implement some animation that Jane made.

Inspired by the artwork of Keith Haring, it was decided to create a thematic side for the 3rd painting and the Egyptian statue to be used. The animated characters would be implemented both in the painting and added as animated texture on the statue. Watching this video, we also were inspired to add some sounds to the scene and maybe even the music accompaniment as it is in the video.

We started this week off by scheduling and holding a meeting with Christos to get the idea of the project approved and any further advice that would help us to formulate the plan better. It was concluded that the footage would be obtained by the tutor, filmed in one of UAL Halls, meaning it was our objective to created as much detailed previs as possible for the shot.

It was essential for to make a list of all the assets and props we wanted to use, which ones would have to be modelled and which we could get as CGI models online. We also had to narrow down what it was specifically we wanted to appear, happen, grow out or over and what statues we would be implementing in the shot. We were given an idea to search for second hand picture frames, roughly of A1 size. After two days going over Ebay and online shops, whilst Jane checking actual vintage stores and exploring the idea of using frames supplied by Giulia, we decided that those would need to be modelled.

For a great shot with good placement of the assets we needed to understand the best conditions and requirements for tracking and exporting that information to Maya, so we were advised to schedule a meeting with Dom, who gave us few sessions on 3DEqualiser last term. Contacting him in Discord, we managed to set the meeting for Friday, before which I had a few goes at rough camera movements in Maya.

At first, we had to decide on the lens to be used for the filming, so I made a comparison video of a 24mm and 35mm lens. As we are working remotely and don’t have the exact measurements of the room, it would be on a safer side to use a wider lens. Dom pointed out that it was necessary to have the floor in the shot at all times, as it is important for the software to know the ground position for accurate CGI placements.

Our 3rd team meeting took place of Wednesday, when we all came back together noting the progress that was made. I received the comments back on which lens they preferred, which camera movements they liked and which weren’t as good. Giulia showed the 3D statues that she managed to acquire, talked about the problem of a triangular mesh and gave some ideas of how to resolve those. Jane also found an Egyptian statue which we all liked. We then thought how to implement the idea of parallax movement inside the pictures into the scene and noted down any questions we had as well as whom we should be asking those questions. I had an idea, that it would be placed a following way into the scene in Nuke, but wasn’t too sure. We further discussed on how would a 2D animation be placed on an object like a statue and had a concept of animated texture explained to us by Giulia.

Upon further exploration of what could be placed in the shot and how it should be constructed, Giulia mentioned an art style called Vaporwave and we though of would could be potentially done to maybe implement it in the project, be it creating new lighting sources in Maya or adjusting the the colors through various passes of the statues AOVs. Another idea, inspired by one of the inspiration posts from Instagram, was to add particle movement around the base of the statues (https://www.instagram.com/p/CGgCLfNJw1M/?igshid=1jfq3fz5thdgi), which we all noted down as a question to ask Medhi, next live Q&A session.

The meeting with Dom on Friday was very informative and helpful as it gave us a better in insight at what makes a much better tracked shot. The advice he has given was targeted at markers placement, how many and where they should be positioned, the actual camera movement and how it was important to make the shot simpler and shorter. He pointed out to which parts of the previs could be re-made and suggested we have a meeting next week, after we gather the footage. I attempted the 4th time at the previs, following his advice and now animating Camera and Aim, rather than just Camera (to avoid the crazy, exaggerating movements appearing at 17th and 25th second), got to the most recent version. The simple 3D objects were also replaced by cards and locators, which are representing the markers. The shot was also cut down to roughly 40 seconds, instead of a minute and 10 seconds. To fit the time, we decided to scrap the idea for a looped video focusing on a simpler but more effective shot.

Given all the placements and position of the assets, I also created a rough plan with all the supposed measurements, given that it will fit in the physical space.

Last week, Jane, Giulia and I had a Zoom call together to discuss and plan the idea for our collaboration project. Being all VFX students, we were inspired by 3D and 2D projections of small animated objects in a pre-filmed 2D shot. As the main reference we took this video, from kidmograph account on Instagram:

The idea was to film or create an Art Gallery room, where the viewer (i.e. the camera) is taken around and shown various paintings, which have this parallax effect as the angle of perception changes, and 3D sculptures which have organic material growth around them. As the camera moves around the room, it ends by coming to the point of entrance where the final frame would loop in with the first frame of the video. I also wanted to put some particle movement, such as burning fire or flowing water in an enclosed space, such as museum glass display, quite similar to this post.

https://www.instagram.com/reel/CJxP9rJpOaT/?igshid=1llhuwvwttulp

When meeting for the 2nd time, we all have shown the visualization tasks we did as well as complied a document with all the resources and references we found. Whilst Giulia came up with a rough previsualization picture of the look of the room, Jane put up some 3D projections of a Vincent Van Gough painting in Nuke, to show that changing perspective. On my end, I created a rough storyboard of the camera movement in the room.

After the second Zoom meeting, we noted all the possibilities of acquiring the footage of a museum hall. The various ideas we came up with were:
1) seeing if a Tate modern was open and it was possible to arrange to film inside it;
2) finding a big hall with an access so that we could make it look like an art gallery room;
3) filming inside UAL;
4) using some stock footage;
5) getting a VR model of a museum hall;
6) OR modelling the entire hall ourselves.

We then scheduled a call with our tutor, Christos, to check all the possibilities with him and get any advice possible on the workflow of the project and how to do certain things, for instance camera tracking, which software to use for it, how to use that tracking information in Maya for models.