Nearly Final

Here's the piece in it's pretty much final form:

I need adjust the audio levels a little still, and the positioning/timing of the titles, however this is pretty much it.

Qupdate

I have been working a little more on the film, however it's mostly been putting together the titles, fine tuning the audio and adjusting the colour correction.

I have not been having fun with the titles, since I decided to go ahead and create them using particle effects in Maya.  Which caused me no end of headaches with my assigned particle IDs not persisting between sessions.  Of all of the pieces created, only one has made it into the final piece, and that is the one for the main title.  Oh yes, I came up with a name finally:

And here is how the subsequent attempts at titles appeared, with an unsatisfactory uniformity:

Then, a couple of days ago I stumbled upon this tutorial and realised that I could have done the whole thing in After Effects in less time.

So indeed that is what I have done for the beginning titles in this video here:

I'll upload a video of what is the piece in it's all but final form soon.  (Again, as I had but the upload corrupted somewhere along the line, and it takes close to an hour to do so).

Outro Wall

The wall featured in the outro shots was recreated but using the original cutout as a guide, and placing brick geometry behind it, matching them up with the existing pattern.
In order to make this process easier, and to ensure that the rendered bricks still framed the footage behind the wall well, two image planes were attached to the camera as seen below:

The front image place contained the cutout, and it's depth was positioned according to the brick wall geometry copied from the Wall to Andrew shot.
The rear image plane contained the footage shown behind the wall (with the original cutout still visible to ensure the positioning of the image planes matched up.
Then it was a case of duplicating, positioning, and scaling bricks in order to create the effect of a broken up wall. 
Having a static shot, close to the camera has required me to increase the amount of detail in the objects.  However since the shot, and the objects are in fact static, this was much easier to do in post production rather than increase the texture/geometry detail.
In order to accomplish this, I imported a photograph of a cracked concrete wall, applied an overlay blending mode, then cloned/rotated areas in order to match the texture up with the rendered bricks.

It Makes Noises Now

After searching far and wide for an online source of free sound effects, I eventually stumbled upon this.  Which has been an incredibly useful resource.  I would have recorded my own audio, however I have the necessary equipment, or the funds to get said equipment (funds in this case referring to the train ticket that would get me to the media loans department).

While the site doesn't have an exactly comprehensive list of audio effects, it has enough to allow you to find audio that will fit with a bit of editing.

The bricks, for example, contain the sound of rubble falling, thunder, a gun firing and a mountain lion roaring at various points.  All happen at once when the big ball of bricks comes hurtling out of the wall.

The rubble effects are the ones that I am most satisfied with.  The swooshes when Andrew meets the wall and the dirt hit sounds need refining.

The swooshes need to be soften, and the sound varied.  I did attempt to vary the audio using the pitch shifter effect in Premiere, slightly adjusting the pitch for each occurrence of the sound, however the audio wouldn't render in the preview or when encoding.  I haven't yet figured out why it does not work.

The dirt hit effects currently sound a little too wet, but with everything else happening in the scene, I could possibly get away with simply reducing the volume of them.

Last post, I lied.  There was at least one more bit of 3D work to do, and that was for the outro shot.  As you can see in the video up top, it looks more like a cardboard cutout than a wall.  It was always intended to be a placeholder, but I either forgot about it or got lazy.

I'll post about how I created the scene later, for now, here's how it looks:

3Done

The matchmove and 3D work for the last shot is now complete.

The solve for this went exceptionally well with automatic tracking, I would put this down to the large amount of rotational information alongside the movement.

the geometry was reconstructed within boujou, then tuned to match the area and fix geometry errors within Maya.

The particle effects were creating by starting by creating a ball of particles around a newton field, then setting that as the initial state.  The first attempt to do this was by filling a sphere with particles, however the system I was using for randomising the instanced geometry was dependant on assigning a random number to each particle at the time of creation.

The ball was then exploded out of shot using an airfield, with gravity turned off for the first few frames of the explosion to facilitate a wider range of movement in the particles.

The final brick flying past Andrew head was animated along a motion path, adjusting the keyframes of the path in order to directly position the brick as his head turns.

Once rendered, and with the footage bought into After Effects Andrew needed to be masked out as he passes in front of the particles.  This was achieved through assigning a number of colour keys sampled from the grass, and a luma key to catch the darkest areas.  The mask is not 100% accurate, however given the amount of motion in the scene the inaccuracies are not noticeable.

In order to make the cut between this shot and the previous, "Wall To Andrew" shot, I used the frames where the ball of bricks entirely fills the shot, rendered the frames after it with transparency and timed the explosion of the bricks to the cut.  There is a noticeable jump in the position of the camera, but the combination of the cameras motion and the timing of the explosion meant that the experiment worked.

With all of the 3D work now complete, aside from a possible bit of re-rendering, or some additional two dimensional effects, the only work left is to fine tune the editing and add in more audio.

Here is the piece in it's current state, complete with updated Wall Pull (added fluid effect) and Outro (added background bricks, rendered from the Roll shot, but from a different angle.  The masking for this was a bit more complicated, as the shot was static.  It initially uses a difference matte to capture the majority of Andrew, touched up by hand painting problem areas frame by frame).

Update

Rather than talk about the piece as a whole, I deemed it more appropriate to write about the work I've been doing on each shot this past week.
My intention is get finish off the majority of the 3D by the end of the week, and spend the rest of the time fine tuning the compositing and editing and getting the audio in.
The plan is mostly on target, the two major vfx scenes are finished and rendered, leaving the rolls (which do not require perfect matchmoving, as objects will not be colliding with anything, and I can reuse dynamic effects used in other scenes) and the post-roll stand up shot, which will incorporate bricks dropping and littering the background.
I would be totally on target if it wasn't for a couple of irritating snags.  More on that later.


Wall Pull Shot Progression
Scene Reconstruction
The scene was reconstructed using the locators from the Boujou export as a guide.  The floor was recreated using my briefly tried and tested technique of evenly placing passively colliding rigid-body spheres on the ground locators, then dropping a fairly stiff simulated cloth over them. It's long-winded but it works.
The wall was quick and simple to construct, with the brick placement and texture based directly on this photograph taken at the shoot:

After a little perspective correction in order to straighten the lines to make UV placing easier, the picture was cropped, resized, and ready to be used as a texture.
Each brick has the same texture on each face, the reasoning behind this was to allow making a variation of different bricks a quick and easy process by simply dragging the UV coordinates to a different part of the brick texture.


This is the finished wall in it's entirety:

The next stage was to take the wall, and dynamically simulate it's destruction.


Dynamic Simulation
A selection of bricks in the centre of wall were made into separate dynamic objects, leaving the outer edges to be used as a passive collider.  The bricks made dynamic are highlighted here:


The process of destroying it was simple.  The activation of the dynamics on each brick is carefully timed with the action in the footage, so each "pull" turns on a few bricks.
The bricks themselves are flung out using a powerful air field situated behind the wall, and angled toward the Andrew's position in 3D space.
This is the animation of the wall on it's own:

All that was then needed was to fine tune the timing of the rigidbody activation, the power of the air field and the collision properties of the bricks in order to get the collision with the floor correct, then the simulation was baked and ready to be rendered.


Rendering
I've made the decision to render the majority of the vfx work using 3Delight.  There are a number of considerations behind this.  The main reason is that motion blur renders a lot faster than with mentalray.  Given the amount of experimentation with getting the motion correct in this piece, rendering speed is a high priority.  Other reasons for the decision are that setting up render passes and associated file destinations is a faster and easier process. I also find it's use of sets for setting up render layers to be far more intuitive and stable than relying on Maya's system alone.


Compositing
For the most part, the composition of this is simple.  These are the following layers currently present:

• Adjustment (for preliminary, overall colour correction)
• Footage (with a mask for foreground objects)
• 3D composition (contains grass hit effects)
• Brick render pass (dynamic and static, color corrected to match footage)
• Brick shadow pass (colour corrected to match footage lighting)
• Hole mask (used to darken the area where the hole is made)
• Footage (main, background footage)

As the rendered material was textured using a photograph taken under the same lighting conditions the main footage, the colours neatly matched up.  An Exposure adjustment was added to the rendered bricks, pushed up a little and that was all that was required.
To avoid having to mask around Andrew in the few frames where he is in front of the wall, the wall fades in a few frames before the dynamics occur.  Given how well the colour and pattern matches, the fade is unnoticeable.


The 3D composition layer uses camera animation imported from Maya, and is currently being used to the effect of the bricks kicking up dirt and grass.  This adds some much needed physicality to the scene.
The effect itself was created in Maya using a combination of particle effects and geometry instancing.
So far there are only two variations have been required, as the effect is subtle enough to not appear repetitive.  These are the animations, one for a hit and one for a skid:

I decided to incorporate these effects in the compositing process, rather than render them all in the scene as the process was faster to implement and make changes to.  Rendering each effect in scene would have had no advantage in this instance, especially given the simplicity of them and brief period they are on screen.


The following is the current state of the shot and vfx breakdown of the scene geometry and cg components in action:

Wall To Andrew Shot Progression

Matchmoving

The footage for this was far less effective than I had originally hoped.  My initial thoughts were that there was a lot of parallax information contained in the background trees.  What I didn't take into account was that the mass of crossing branches created a constantly changing pattern.  Even after placing a number manual tracks the solve still had large problems.

Fortunately, the frames where the wall was in shot matched well and only in the subsequent parts the solve had issues.

After the wall is out of shot, a perfectly accurate solve is not necessary, as no objects rest or come into contact with the ground.  All I need for this shot is the translation and rotation to match up well enough to look convincing.

Due to the poor solve, some frames of the camera animation jumped excessively, a couple went a few hundred units away and faced the wrong direction, and toward the end a fast backwards motion occurred that shouldn't.

The camera animation was fixed through a process of editing the animation curves in the graph editor, smoothing out animation where needed, deleting wildly inaccurate key frames and scaling down the curve that caused the backwards motion.  During this I discovered that as long as the position of points on the curve retain their relative positions, large changes can be made to adjust the animation.

While time consuming, the manual adjustment was far more successful than the trial and error method I was using in boujou.

Bricks

Since this shot occurs immediately after the Wall Pull shot, the broken brick wall and individual brick assets, as well as the associated texture were reused, negating the need to replace and reposition them.
This shot also has a brief, dynamic brick blast, using the same techniques as the Wall Pull.  however, the centrepiece is a giant, murderously twirling ball of bricks flying out of the wall.
This effect was achieved through the use of particles and geometry replacement.  The particles use a similar air field to the Wall Pull shot for the initial blast, however the ball itself is held in place using a newton field, moving along a motion path carefully keyed to keep it in shot.
Parented to the newton field are a vortex and an air field to create the swirling motion and turbulence.
Even with those effects occurring on the particles, the ball still had a tendency to clump up and appear static.  I solved this problem by placing a couple of spheres along the motion path to disrupt the particles as they crossed over them.
To create a decent amount of variation, eight different bricks are using in the instancing, with the geometry initially positioned at varying angles in order to counteract any uniformity that may occur in the simulation.
In the end, this took far longer to create than it should have, as the scene became corrupted somewhere along the way, causing me to have to change the path and reattach the cached particle animation every time I opened the file.  This took me a lot of crashes and a couple of hours to figure out.
Using the same fields, and with a few parameter adjustments, fluid effects were used in order to create a dust cloud emerging from the wall behind the ball of bricks


Rendering and Compositing
To avoid having to render out both the static wall and particle bricks whenever I need to adjust the animation, both were rendered out separately.  The particle brick render pass uses the static brick wall with a surface shader applied, and zero matte opacity in order to use it as a mask for where the wall should be.

Once rendered and composited, it became apparent that areas on the bricks were being hidden from both the directional and ambient lights in the scene.  Therefore faces on the bricks became black, which looks incredibly out of place in the scene.  


This was fixed by duplicating the layer in After Effects, applying a luma key to key out anything lighter than the dark parts, than applying a level adjustment to lighten the layer.
This allowed for a quick fix without the need to re-render then entire animation and universally adjust the whole layer.


Additionally, a few bricks that hit the ground were visible behind the ball.  Given that the solve is being used without 100% accuracy, and that the landed where they should be in shadow of the tree, these bricks were very out of place.
Since they were only visible for a few frames, and only took up a small portion of the screen; the quickest way remove them was to hand paint a mask frame by frame.


Here is the current progress with this shot:

Progress, Eventually.

After spending the last few weeks catching up on other projects, it's time to resume cracking on with this one.

Last week I ventured out to capture some footage with a mind to incorporate it into the final piece.

One of the things holding me back from making progress on this film has been the planning stage.  Planning is in no way one of my strong points.  Arguably I should be pushing myself into the process, however at the point where it's holding back a project, my best bet is to play to my strengths and improvise and be inspired as I work.

The following footage is the result of the shoot:

It may be very raw and poorly edited, but the main purpose of this was to work out which scenes would require match-moving and which would need reshooting.

The next stage is to take the parts that will require match-moving, then take them into Matchmover/Boujou in order to test the effectiveness of the tracking.

(Currently, I am testing the footage in Boujou, as it handles motion blur far more effectively than Matchmover.  Also, it's masking and manual tracking tools are far more intuitive to use.)

Once loaded into boujou, the first step was to mask out Andrew in order to prevent boujou from tracking him.

A few attempts to solely use automatic tracking were met with limited success.  There was a lot of slipping around the frames where the camera movement was particularly harsh.

Due to this I incorporated a number of manual tracks, picking points that I could make out when the motion was particularly blurred.

Using the manual data I then ran another automatic track, which resulted in a successful 3D solve:

This is the main shot that will contain matchmoving, a couple of others will too.  Particularly the post-roll shot.  Then preceeding rolls are so short, that matchmoving is possibly not necessary, and I could simply represent blurred bricks hurtling across the screen.  If that method is successful, then I will do so.  If not then matchmoving it is.

Next post I'll go into more detail about the film itself and the considerations behind the shots and editing.

Two Weeks In The Making

There's been little progress in the last two weeks on this project, so it's high time I cracked on with it.


First off, I'll analyse why the footage taken a couple weeks ago was not successful, and unusable for the most part.
Then I will go through a second attempt at shooting, and how different problems were dealt with.


Out of the four pieces show in the last post, only one of them was even moderately useful for tracking.  This one:

The sideways motion, coupled with the depth of the location resulted in a decent amount of parallax information. However the motion blur present caused the tracking to completely lose the position and pattern every few frames.
Some useful information came out of this regardless of the blurring issues.  Patterns tracked effectively on the bricks whenever the camera motions were small and shadows can be used providing the light is constant and the shadows sharp.


The following shots were useless:

The first two were almost tracking along a flat plane, so very little parallax information was available.  Plus the combination of the majority of tracking points being a repetitive pattern and the motion blur caused the auto tracking to be impossible, and manual tracking more trouble that it was worth.

The last shot used a camera rotation that was too fast and not really any movement.

Similarly, this shot contained almost no parallax information for the majority of it.  And again, motion blur caused what little information there was to become useless.

On the other hand, the above piece worked very well, despite the blur that caused some slipping of tracks, the content and movement created a large number of points to be tracked and the track came out usable with minimal fixing and slipping.

The results of the successes and failings was taken into account and the main issue that needed to be taken into consideration was the motion blur that was destroying all the tracking information.

By default the cameras appear to have a shutter speed of 1/25 second (when shooting at 25fps).  So regardless of shooting either indoors or outside in bright sunlight the motion blur was the same.  The same amount of movement was being recorded despite the significantly varying light levels.

The only option seemed to be finding the shutter speed settings, which I found on my third shooting attempt.  Hidden away somewhere not in the main menu and could only be found by touching the screen.

The first batch of test footage was shot with the shutter speed set at 1/2000 second.  The results were somewhat jerky, however a lot of detail was retained in almost all camera movements.

The next step of this will be to find a sweet spot that retains enough detail while allowing at least some motion blur.

Another method tested to control the motion blur was to use the tripod as a counter-weight for the camera, effectively creating a cheap steady cam.

This smoothed out some of the hand held movements, however it had a downside of being extremely cumbersome, and very difficult to both shoot and walk with.

As the steady cam rig training is currently unavailable, my best bet seems to be to either refine this method or use a dolly.

The other issue surrounding the first batches of footage was the narrow field of view present in the cameras.  Not only does it make it difficult to get a subject entirely in shot and frame it decently, but it also accentuates the movement of the camera.  Which is less than ideal.
Given that no other lenses are available to use, the only options are to A:  Shoot something smaller or B:  Shoot from further away.


The above fixes were taken into consideration when it was time to shoot some more footage.
The results are as follows:

The bridge footage was successful for the most part.  It turns out that grass is a wonderful surface to track as it's full of unique patterns, and if there are lighter/darker patches as well as a few twigs and dead leaves lying there, plenty of contrast too.  The bridge itself wasn't too great, as for the most part it's a uniform colour, apart from a few blemishes and the patterns on the corners change shape a little too much.  But the path and grass worked very well.

The tree shot worked largely because of the sheer amount of grass present, as well a number of points on the tree, branches and background.  All helped hugely by the almost complete lack of motion blur.

I was initially worried about the vehicles moving in the background, especially the lighter ones.  MatchMover did attempt to 2D track them, but disregarded them as bad tracks during the 3D solves so they were not an issue.  I can imagine that it would be possible for them to move in such a way that they could be seen as parallax information, so that's something to take into consideration for future shoots.  The important thing is that things can be moving in shot, just as long as they're moving in the wrong direction.

Building the scene for the bridge composite was simple, I simply used a few points around the path tracks and constructed geometry around them, using the background shader to mask around the edges.  The bridge was bright enough to key out with a simple luma key, with a polygon placed in the 3D scene to counter a problem area (a solution that seemed easier and quicker to implement that keying a mask in After Effects).  The dust could effect didn't quite come out as intended since I assigned a texture time variable the frame number, rather than the clip time.  However, for a test shot it's not worth fixing, especially considering the render time of fluids.

The tree composite required a bit of experimentation in order to recreate the uneven ground, but the technique was quick to set up and successful.

In short, rigid body spheres were placed evenly at the position of trackers on the ground, then a dynamically simulated cloth was dropped on them in order to create a surface that conformed to the shape of the tracking points.  I'm sure there's an easier way to do this, but considering how well it worked, I'll stick with the method for now.

Then it was simply a case of spawning a bunch of particles and rendering them out.  They were lit using an area light, with the colour sampled from the sky present in shot then rendered out with both that and an ambient occlusion pass.  In the colour render the ground used the background shader, and an ambient occlusion shader in the related pass.

Next up, I need to figure out exactly what I'll be shooting for the final piece.  Chances are it will involved particles rolling around on the ground.

On The Wrong Footaging

Finally got a chance to shoot some test footage, after making the decision to use the environment manipulation idea.  The reasoning for this largely being that it would be considerably easier to find a location and control the shooting of a wall over a road.

Unfortunately the test footage isn't particularly useful, as there is far too much camera shake present.  Looks like I'll need to find a way to keep the movement smooth.

Despite these short-comings, I've begun to put together a few animation test for the kinds of effects to be used in the film:

Progression Pants.

On the day I set aside to get outside and shoot some real test footage, it ends up raining.

Therefore the real shooting didn't happen, so instead here's some dodgy indoor footage.

The kitchen and cage footage were moderately successful, the wall footage not so, as I accidentally cropped it to a point where the camera was just rotating.  Unfortunately to actually get the entire wall in shot required me to stand in a part of the room where there was no space to walk.  The ridiculously narrow field of view on these cameras is somewhat limiting.

Setting up the coordinate system within MatchMover is a total pain, and it's proving to be quicker and more reliable to rotate the whole thing by hand in Maya.  Decent visual feedback helps a tonne.

Here's Walter with his opinion on Matchmoving:

Concepted.

So far, so conceptualised two ideas, so here they are:

The first concept very much speaks for itself.  It's a chasm in the middle of the road.  The second image is based off of the idea of manipulating the environment, either with of without an actor taking the role of the manipulator.

The chasm concept would not involve dynamics, solely the chasm existing as a feature of the on the environment.

However the manipulator concept would feature a large amount on animation, albeit on smaller objects.

I've yet to weigh up the pros and cons of both of these ideas, and could quite possibly end up combining them if seems feasible given the time constraints.

With regards to experimentation, I have so far discovered that poorly lit and out of focus footage is next to useless, and that averagely lit, yet in focus footage is usable, but far from perfect.

Hopefully I'll get a chance to film some footage outside when it's light this week.