We had to practice the tracking node in Nuke for this weeks exercise. Using the footage shot handheld during the trip to St Petersburg I had last year, I practiced applying this instrument and inserted some checkerboards into the various places that were tracked.
Firstly, I stabilized the footage using 3 different points at different depth of the shot. This reduced the warping effect that occurs sometimes when applying this node. Afterwards, I chose three different windows which I would be applying and attaching the checkerboards to, finalizing down to the left, middle and right windows. The main problems that occurred were with the left window, specifically for the few seconds it would disappear from the shot: the checkerboard would not update the correct scale, as the shot is being zoomed out. I solved it by adding the keys at various frames for the CornerPin node, however, it still feels that the checkboard is changing its size incorrectly. With the window on the right side it was harder to find good and clear points, so I applied extra contrast via the grade node to the plate the tracking node was used on. At some frames the corners are jumping and I tried fixing it by adjusting the tracking keys by hand, however, some of them would move incorrectly, so it was left as it is.
Tacking node in Nuke window
Nuke script
For the second video I used this cool shot of the rotating ceiling followed by downward tilt. Having tried applying the tracker node for paintings on the ceiling, I decided to use the planar tracker specifically for the frames where the camera was rotating, followed by a normal tracker in the frames where there is little rotational movement. It was hard to track the 3rd point (first bottom right, then top right) of the first picture that we see. There is still movement of the checkerboard that I don’t like, but I am not sure how to solve it, as I tried re-tracking it using various points for a few times.
Checkerboard in the footage
To enhance the tracking abilities, I applied grade and sharpening nodes to the denoised plate before using any tracker nodes.
Further practice of colour correction on various still plates. This time I chose to use the same rotoscoped picture of a hobbit, Bilbo Baggis, captured in this specifically composed shot with such a camera placement which could have been easily placed in a lot of other shots of a very similar perspective. I practiced with other 3 various plates from different moves and quite different lighting conditions.
Original picture
Merging with this scene from Avatar was tough as the colours are quite blue and cold, feeling quite extreme sometimes. It was challenging to get the correct skin colour tones, with the reference used was the exposed elbow of the person lying in the middle of the street.
Hobbit crossover with Avatar
It was easier to match the colour of this shot from Jurassic Park.
Hobbit crossover with Jurassic Park
When adding the hobbit to this frame, to add more realistic feel I added the shadow as well, but it still feels like it could be stewed more in the X direction. I also had to re-try the colour correction in order to get the skin tone correct.
Hobbit crossover with MIBNuke script for Hobbit and Men In Black
First version of mask taken from MudBox into MayaProgress with overall shape, horns and teeth
There is a lot of integration and interchange between Maya and Mudbox, which helps with easier application of the tools and direct practice of both programs. The objective during the next week is to finish all the details such as teeth and adjust the face topology more correctly (such as fuller lips, mouth and eye shapes, better definition in the eyebrow area), find a reference picture for details and colouring to be created further in MudBox.
In the 2nd part of the lesson during week 3 we started learning about head scan and using a CGI head model. Understanding the topology of the head and how to create a model for a mask using quad-tool in Maya was something we focused on. It is important to figure out and follow the correct way to build the topology such that further down the pipeline, if you had to animate it, the joints would bend correct way, the muscles will be build naturally and all the movements will seem to be as realistic as possible.
Head topology edge flow
Cartoon head topology
Using a 3D model of a head we build up a mask for half of the face, which we then mirrored and created the following.
A rough shape for mask.
The file was then taken to the MudBox, as on week 4 we started working on adjusting the mask and bending it to fit our inspiration for a kabuki/demon mask. This is the result I got by the end of week 4.
Version 1 of the demon mask.
The following link shows the mask which I am using as my reference.
For the week 5, the exercise was to roto out an image, white balance it and colour grade it correctly to fit in the scene of the second image. I practiced with both still pictures as well as moving picture.
For the stills, I chose this picture of Captain America and decided to put it into this image from “Indiana Jones and the Last Crusade”.
Captain America from “Avengers: Endgame”Indiana Jones and the Last Crusade
First, I cut out the roto shape, which I then white balanced and colour graded using the information from the second image.
RotoscopeWhite balancedColour graded
Due to the position of the legs closer to the camera, I had to place the objects in the space between the legs and the background from the second shot. In order to do that, I transformed the colour graded image of Captain America to where I wanted it to be positioned in the second shot and merged together with the roto of blocking subjects in the 2nd shot using the Merge(out) node.
Transposed and cut out roto of Captain America
This is how the final shot turned out.
Final composition
Looking at the final composition, addition of the shadow falling towards camera would have made the shot appear to be more realistic. Below is an image of how the Nuke Script looked like.
Nuke script
In the moving shot I used this poster of Uncle Sam and placed it instead of one of the pictures on the wall, following a very similar path as for the still images.
For the 4th week our coursework consisted on rotoscoping about 60-80 frames worth of an organic movement in the shot. Here are the original, green matte and with gray background rotoscoped shots.
Original shot
Green matte
Grey matte
I included the shapes around baby hairs so that I could apply the luma key node later during the course in order to isolate the dark hairs against the contrast of lighter wall. Following from the comment by a tutor, I worked on adding feathering to my roto shapes such that they would not look as sharp as before.
The toughest part of the shot were the movements of the shirt material with all the folds in various places. I found the arms to be easiest as I had a better understanding of how to divide the shapes between various places, after I had some practice with head and the body.
Nuke nodes
Roto shapes for head
Roto shapes for body
Roto shapes for armsRoto shapes for furniture
For my second rotoscope, I used the footage taken earlier in week 2, where I rotoscoped out the running girl at the last frames of the shot, i.e. when she is passing by the camera. I found getting the correct feathering for the coat the hardest. Another difficulty was drawing roto for the cap as it would occasionally be against the white background and it would be hard to see the outline.
Gaining more and more popularity, the series game Assassin’s Creed has united people from various background over the love of both game, history, design, style and storyline, especially the last few games being set in Ancient Egypt or during the time of Peloponnesian War between Athens and Sparta. Whatever the time period, the artists would conduct a careful and an informative research of the clothes, architecture, interior design, design of day-to-day objects and everything that would submerge the player back into the time period they are playing in.
For the Assassin’s Creed: Odyssey, set during the 431-442 BCE in Ancient Greece, with the consultancy from of Dr. Stephanie-Anne Ruatta the Ubisoft team travelled to Greece to take notes, pictures and sketches of the buildings, to get to know topography, archaeological sites and to visit museums, in order to understand and collect the knowledge of the country during the period of Classical Era. Once all the information was obtained and the research conducted, the designers built the map onto which seven different biomes were successfully integrated. Examples of sub-biomes involve snow-capped mountains, marshlands, clay form and burned forests as they add visual diversity and give each region its own unique characteristic and the purpose it served at the time.
Dominic Gladu-Despatis – Team Lead Artist – Assassin’s Creed: Odyssey
To add the details to everyday life of the people based in that time period more than 4,000 unique props were created, such as clay jars, baskets of fruit, ropes, furniture, tents, scrolls, chests. To get around the time-consuming task of placing the props around in the world, the team would build individual component that would already be populated by the props, such as table with jewellery or a basket full of fruit, which further could be mixed and matched to be placed in various sceneries. As the game is based during a war, each city state had to have its own army, which is allied with either Sparta or Athens, so attention to their uniform, weapons, philosophy and behaviour was important.
Details such as textures play a big part in design for families of assets. Each artist was in a charge of a different kit of materials using different software, making sure that their own materials were coherent with the needs of modellers. In the pipeline of the game the modellers created 3D assets, Level Artists placed those to build the world and Materials Artists who created the materials for those assets.
Philippe Routhier – Level Artist – Assassin’s Creed: OdysseyMiguel Bouchard – Storyboard Artist – Assassin’s Creed: OdysseyEarly sketches for AlexiosFred Rambaud – Concept Artist – Assassin’s Creed: OdysseyMaxime Lariviere – Senior Environment Artist – Assassin’s Creed: OdysseyJimmy Malachier – Lead Prop Artist – Assassin’s Creed: Odyssey
One of the game features that require more thought and careful attention in pre-production are the cinematic scenes which would portray interaction between the main character and historically important figure or show the dramatic event in their life that would lead to a specific outcome of their fate and where they are at right now. One of the most powerful scenes of such that I found was the flashback, when playing the role for Kassandra, of how she was thrown of a cliff by her father. As an audience we see the desperation of the mother as she is trying to stop her child getting killed, the impotence of Nicolaos, the father, as he believes in what has to be done by the placement of the camera angles and composition of the shot.
The sharp transition from him kneeling in front of the child and appearing heartbroken to taking her off the ground and holding over the cliff intensifies and highlights the change in his mood. From caring father, to the ruthless and harsh leader that he was, following the strict order of executing a punishment, the scene is only intensified by the storm that is happening on the background. All the details found in the scene remind us of the importance of gods’ presence in Greek culture: from clothes, priests, statues, to the lighting strike as Nicolaus releases his grip and lets Kassandra fall (maybe presence of Zeus, or resemblance to the god). Those are constantly present throughout the game, as they were intertwined with the societies beliefs and way of living the life at the time.
Assistant technical director (TD) is an entry level job, which concludes the tasks of helping to identify and fixing problems as well as ensuring of providing and supplying everyone in a VFX pipeline with the tools that are needed. They have a great understanding of how the pipeline within VFX works and the various job roles that are there, from stages of pre-production, through shoot, editorial, VFX to grading. They also need to have an understanding of the software used by the artists and be aware of the needs and limitations of various departments in a production. The assistant must gather information across all the departments to learn of their needs. They should be able to design the solution for arisen problems and have some coding skills to create small-scale tools using the established employer workflow requirements. ATDs will be utilising a variety of the industry standard graphical applications, scripting languages and operating systems.
Dealing with minor bug reports is part of their tasks, thus helping the TDs to focus on much bigger problems, for instance rendering errors. The responsibilities that fall on the assistant TDs also include data management, archiving, restoring and tracking data, converting and resizing files where needed; they keep the project on schedule. Other than the pipeline TDS, they work closely with the research and development teams, who design and test any new software.
While undertaking the job, the assistant must have good communication skills, as they should be able to interact well with the variety of staff at different levels and be able to understand their needs and assist accordingly; have a problem-solving mind, think analytically, be able to come up with creative and efficient solutions; have an eye for detail when designing tools and fixing technical issues; have strong knowledge of the pipeline; strong programming and coding skills in Python and C++.
Production Coordinator
The main role of a production coordinator is to aid with the management of the production of the film’s visual effects. They are responsible for the schedule of the production and tasks given to the team. In the project pipeline, the production coordinator start work during pre-production. Working in the production office, their tasks include overseeing what goes on with the phones, photocopier and paperwork when the line producer and production manager are on set, setting up the production office, organising equipment, supplies and staff, coordinating travel, accommodation, work permits and visas for cast and crew, distribution of the shooting schedules, crew and cast lists, scripts and script revisions. During the production stage they should be preparing, updating and distributing crew lists, daily progress reports and script changes, dealing with call sheets and transport requirements, organising couriers and shipping companies. Toward the end of the shoot, the production coordinate helps the production manager to wrap up the production, examples of their tasks include closing accounts with suppliers, returning surplus stock and tying up all loose ends.
The key skills of a production coordinator include knowledge of the filmmaking, as they have to understand the process and need of each department, be organised, as they would have to plan and multi-task a lot, be able to work under pressure, be innovative, be a good communicator, work effectively in a team and know how to budget. They report back to the line producer and production managers. They communicate with all the heads of department and rest of the production department. If the project is big and there is a such possibility, they could delegate tasks and jobs to assistant production coordinators and production runners.
Modelling Artist
A modelling artist creates the models for all the three-dimensional assets required within a project, be it characters, weapons, vehicles, furniture, trees, plant, rocks, animals and so on. Their workflow often starts with a brief drawing from a concept artist or collection of reference materials, such as photographs or line drawing sketches, followed by building a 3D model from that. Firstly, the create a ‘wireframe’, referred to as ‘mesh’ of an object. From that, they should be able to sculpt the object model to the close resemblance of what is intended. Using the digital tools, such as sculpting brushes, physical graphics pen and tablet, they complete their work. They could be specialising in a certain area, depending on the project requirements. Hence there are varieties of a modelling artist: environment, character or vehicles artist.
An environment artist focuses on building the CG world. Their role ca be divided into two parts: asset creation and layout. Both of which require the artist to have a good understanding of lighting and textures. A character artist is someone who makes characters and creatures. Starting with the figure sculpting, where aspects such as topology, UVs, shading are involved, the position of a character modeller is not an entry level position. A vehicle artist would usually work on assets such as cars, spaceships, weapons, robots.
Modelling artist would work at an early stage of the CG and 3D part of the VFX pipeline. After they produce the models, those are further passed on to be animated, given texture and lit. They could be working for a VFX company or be a freelancer. The key skills of a modeller involve the ability to draw, be able to interpret the 2D model and build a 3D model from that, be able to use required software, such as Blender, Maya and ZBrush, be organised and collaborative. They work with the concept artists and pass on their work to the texture artists, riggers or animators.
Texturing Artist
The role of a texturing artist is based around painting surface textures on animated characters, environments and props. Examples of the textures that the artists work on include wrinkles, fur, scales, sweat, mud and sometimes even one that would not be found in real life. In this case the artist would have to invent new textures using their imagination and creativity. Their workflow starts of with a 3D model created by a modelling artist that is usually a plain grey shape, with which they will be further working by adding details onto the surfaces until the model will look like a photograph. Once they are happy with the result, they can ‘bake’ the texture from one surface so that it could be used elsewhere.
Texturing artist could work for VFX company or studio or be a freelancer. A set of skills that they should be good at include having an artistic eye and understanding of form, colour and texture, recognise what makes an image appear realistic in terms of light, colour, composition and perspective, be good at photography, have a knowledge of VFX programs they will be working in, be organised and collaborative. The software that a texturing artist will be using are Photoshop, 3D Paint, UV Layout/Editing, RenderMan, Mental Ray, Maya, Shaders and Houdini. There is a variety of tools involved in the job, such as software, platforms and rendering environments for textures to be created. Texturing artists also use digital matte painting techniques to create textures and they work with advanced surface types, subsurface scattering and global illumination.
Little Shop of Horrors (1986) is a horror black comedy musical film. The story is based around a man, Seymour Krelborn, who works in a plant store in New York. He finds a carnivorous plant, as is learned later, which helps him to gain fortune, love and fame, but feeling guilty for all the deaths, Seymour ends up having the plant be killed. The film was based on an off-Broadway musical, which had a much darker ending, where the main character and his lover were killed by the plant. The movie was shot on a film.
This film mostly used special effects (SFX), specifically puppets for the plant, Audrey II. Those were designed by Lyle Conway. Six different stages of growth were made for the plant. Specialists in mechanical effects, Neal Scanlan and Chis Ostwald, were hired to construct the mechanisms required for the operation of the plant. To operate the plant’s mouth a scissor-type lever was hidden within the table the plant was on. The vines were cable-controlled by people underneath the stage.
One of the toughest parts of the plant operation, was to make sure that it was lip synching well enough to the song, but it would often be found to be quite slow. The solution was to film the plant at a slower frame rate and then speed it up in the post-production. When it was in the scene of its largest form, it took about 60 puppeteers to operate it.
In the whole movie, no blue screens or opticals were involved in the scenes with Audrey II, except for the ending where the plant gets electrocuted. Those were designed by Visual Effects supervisor Bran Ferren. Some Visual Effects were also applied in the original ending of the movie, where the plant killed my main characters and took over the world.
Made in 1981, An American Werewolf in London is a horror comedy film which was written and directed by John Landis. The plot is resolving about two young Americans which are attacked by a werewolf. One dies, while the second, David, survives and turns into a werewolf. He later attacks civilians in London, but towards the end of the movie gets shot and dies. The movie was shot on a film. It received a first ever Academy Award for Best Make-up in 1981. The ideas and realisation of make-up was worked on by Rick Baker.
Only special effects were used in the movie. Puppets had to be produced for the first attack of the werewolf, where it was just a head and a bigger size body was made for the attacks in London. It was a werewolf suit built on a wheelbarrow structure, where the operator would lie on a plank and use their arms to propel the creature forward. But the most important part of the movie was the transformation scene which required months of work to get prepared for. For stretching a unique material was used which would later disintegrate. The growth of hair, which Baker said to be the easiest effect to produce, was filmed as the hair was pulled through a rubber-type of material and played in reverse.
The progression of make-up for Jack, the friend that died during the first attack, was also impressive, with every appearance he made his state would deteriorate. For his final appearance in the movie theatre in London, they made a puppet.
Total Recall (1990) is an American science fiction action movie, directed by Paul Verhoeven. It is based around a main character, Douglas Quaid, who gets involved in espionage on Mars and is trying to understand if what he experiences is real or a result of implanted memories. It was one of the last Hollywood films to employ large-scale use of miniature props and sets instead of CGI. It was also one of the first movies to attempt computer generated imagery.
The use of miniatures and motion control technology was implemented during the production of the movie. Company called Stetson Visual Services was hired to create the miniatures, head of which was Mark Stetson. Originally built in the studio, the miniatures would get assembled on the stages at Dream Quest. Most of the miniatures were shot at very long exposure times for believable depth of field, so that a five second shot or each pass would take hours to shoot.
X-ray scene from Total Recall (1990)
The X-ray sequence was captured using the motion capture, adapted from golf industry at the time. A choreography program, Power Animator, together with their own written renderer, modeler and compositing software was used. The movie was shot on film using ARRIFLEX Camera and Zeiss Lenses.
Gremlins is a 1984 movie, directed by Joe Dante, known as a comedy horror film. Written by Christ Columbus, the story is based around a young teenager, Billy Peltzer, who receives an unusual Christmas present from his dad, a Mogwai, which then spawns into other small, evil and destructive creatures, Gremlins. An attempt of trained monkeys to play the creatures was maid, but as the monkeys were scared by the make-up, the team had to create puppets of the little mogwai and gremlins, which were designed by Chris Walas.
Gizmo close-up
Making of puppets
As the puppets were small and fiddley, they would break a lot, so a few copies of them would have to be created. For the close-up’s larger versions of the mogwai had to be created, in order to portray more emotions and facial expressions. In the scene where a gremlin explodes inside the microwave a large balloon was used. To operate a puppet, it would take about 5 people. For the scene where new mogwai are created, it took about guys underneath the table to push out the rubberised fur balloons. The movie was shot on film.
Made in 1989, The Abyss is a science fiction film written and directed by James Cameron. The story tells of an U.S. search and recovery team working on an oil platform racing the Soviet to recover an American sunk submarine. As they decrease further in the ocean, they come across something unexpected. Around 40% of the scenes had to be shot in the water, specifically a retro fitted decommissioned nuclear reactor facility in South Carolina. 7.5 million gallons of water were pumped into large tanks. For the scene where the two main heroes, Virgil and Lindsey, collide with the main antagonist, Cofey, in the submarines miniatures of those, about quarter in size, were created, in order to eliminate the need in damaging the pre-build machines. In the shot where the dome appears to be cracking, sellotape backlighted from inside the dome was used.
Other than using physical effects, the movie also used some CGI. Industrial Light and Magic (ILM) worked six months for 75 seconds of computer graphics of the water tentacle. ILM had to take a lot of storyboard pictures, set reference photos and used, not released at the time, program called Photoshop. A 3D scanning of water tentacle maquette was produced with the Cyberware 3D scanner. To mimic facial expression, the character was animated using commercial 3D software. It required eight scans of Ed Harris’s facial expressions and twelve of Mastrantonio’s. ILM had to design a program to produce surface waves of various sizes and kinetic properties of the creature, such as reflection and refraction. The movie was shot on film, using ARRIFLEX 35 BL Camera and Panavision System 65 Camera.
Made in 1996, Independence Day is an American epic science fiction action film, directed by Ronald Emmerich. After a worldwide attack by an extra-terrestrial race of unknown origin, the story starts of groups of people converging in Nevada desert. With a counterattack, scheduled on 4th of July, is planned around the world the film focuses on the events leading up to the final battle. The film had a mix of special and digital effects applied throughout. Scenes such as opening shot of the moon or explosion of the buildings included miniatures.
One of the challenges was to film the explosion as the fire tends to go up. In order to create the shot where the fire is going down the street, towards the camera, big maquettes were hanged vertically at a slight angle with the camera about 5 feet above. All the explosions were done practically. The production had access to one full-sized working fire jet, which resulted in being re-used for various shots. For all the other jets, they used models in background or FX composite shot.
Defense system of the alien ship
Models and textures for F-18s and Alien Attackers were built by the in-house VFX unit in Alias. Those were later converted to Side Effects Software Prisms and re-built for the pipeline. For the missile trails smoke textured particle spheres rendered in Mantra were used. A so-called ‘Sparky’ toolset was used throughout the production, originally written by Bob Bredow and Pete Shinners. It has a node-based system and a rendering subsystem. The movie was shot on film.
The most recent of movies listed, Blade Runner 2049 is of a 2017-year production an American science fiction fil directed by Denis Villeneuve. During the movie production both special and visual effects were used.
CGI of the hand in and out of sync
The miniatures were worked on by Weta Digital, whilst the work on visual effects was done by a few companies, such as Framestore, DNEG, BUF, MPC, Rodeo FX and UPP. One of the challenging scenes was a love scene between the main character of Ryan Gosling, Ana de Armas and Mackenzie Davis. The DNEG was working on that shot. The love scene was shot with the two actresses, where notes were taken of the first one and her movements, dictated over to the second actress during her take. Four GoPro cameras set up around room helped to capture the movement, as each character had to be 3D-tracked. The movements of each actresses had to be analysed and adjusted accordingly, wherever their movement had to be in or out of sync.
The movie required a fully digital character to be created as it was represented by an aged beyond timeframe the movie calls for actress. Other than a current scan of the actress, a lot of research and reference photos had to be dug up and used from her years of that specific age. The hardest part was to make the acting correct, just as Sean Young did it in the original movie. The movie was shot on digital in 1.55:1 aspect ratio from a single Arri Alexa XT studio camera with Zeiss Master prime lenses, assisted with an attached crane arm or a dolly.
Both film and digital cameras have their own advantages and disadvantages. However the choice between either for cinematography did not come until early 2000s, when Hollywood started to capture films digitally. Around the year 2013, digital shot films were more common than celluloid among the top 100 grossing films. The most used digital camera is made by Arri.
The advantages of the film camera include aspects such as a lower initial cost, the film delivers higher dynamic range meaning there will be more detail in both darker and lighter areas of the composition, it is not as sensitive to subtle focusing issues and exposure problems, it usually has a higher resolution. The disadvantages come with the size of the camera, storage of the film, purchasing and developing film. Whilst shooting, your decisions must be thought through and permanent, as less takes would be available, compared to the digital. With the digital footage is ready to be sent further down the pipeline, the editing or work on Visual Effects can start right away. However, with the film, there is a waiting time between the film being processed and scanned.
The advantages of a digital camera are the lighter weight, smaller physical requirements for the memory storage, immediate review of the shot footage, easier to edit the product, generally a more flexible usage of the camera on a set.
