Hide lateral panel
Monk
0
Monk
Eibriel - 2014
Eibriel - 2014

Monk

Blender + Cycles (no Hair and no SSS) + Gimp

0

Hide lateral panel
Orange Sky
0
Orange Sky
Eibriel - 2014
Eibriel - 2014

Orange Sky

Blender + Cycles + Compositor

0

Hide lateral panel
Pepper Kiss
0
Pepper Kiss
Eibriel - 2014
Eibriel - 2014

Pepper Kiss

Blender + Cycles

0

Hide lateral panel
Inivisible Man
0
Inivisible Man
Eibriel - 2014
Eibriel - 2014

Inivisible Man

Blender + VRay

0

Blending Pipes I - Introduction

I'm starting a serie of blogposts about building a workflow around Blender. I'm currently working on the feature animated film Kiribati, and the following articles will tell you about my experience on it.

What makes Blender so great for small studios facing big projects is that the Blender Foundation build not only the software, but an entire workflow around it. Every film they did test and improve that workflow and we can take advantage of that building our own pipeline on top of it (making some changes to fit a biggest project).

This will save you time and resources; other studios and artists are familiar to it, allowing to share tools and procedures.

This workflow was also designed to allow the collaboration of artists working remotely from their home, and apart from the physical distance you cannot distinguish them inside the pipeline. Also Blender is multiplatform (as every tool on the workflow, allowing artists to work on their preferred OS, being Windows, Linux or MacOSX.

When I was hired to build the pipeline of the movie, I investigate about the work done on other movies, I read about Blender Open Movies ( valuable info on the wiki), the feature film Plumiferos (Blender Conference Talk 2006 - 2007), I see more BConf talks, I've a copy of the book "Digital Movie-Making".

At that time was not decided yet if the movie will or not in Stereo, so I learn about it also. Other thing to decide was the Render Engine ¿Internal or Cycles?.

I'had no experience working on a animated feature film, I had no experience building a complex pipeline, so I read, investigate and lisen people that knows.

The movie is not finished yet, so this articles have some suspense on it ¿Have I succeded on my work? We don't know that yet :P

Presentation

When I started working in the movie, the structure was based on Dropbox and a central server for file sharing, and Trello for Task Management.

The movie was a really small project, with a projection of 20 artists on the peak of production, and a budget of 1% (one percent) of a small Pixar production.

On April 2013, after study the requeriments of the production, I made the following presentation. Some things works as intended, and some things not. I'll be pointing that in the following posts. Lot of things change since then.

Introduction

The technical aspect on the making of Kiribati is based on a minimallistic and simplistic concept. Shortening the intermediate steps to get from Storyboard to Final Render on the less steps possible.

General observations

To achieve this the Pipeline is based on the Pipelines used on other projects made with Blender by the Blender Foundation, and other similar projects around the world. The use of a Pipeline already tested and polished by other productions ensure a fluid production, advancing constantly.

The principal characteristics of the Pipeline are:

  • Centralized system (All information on the same place)
  • Non destructive system (The following step don't destroy the info of previous step)
  • Keep it Simple (Only one file by shot, only one place for everything, only one Editing file from Story to Final Version)
  • Make it right the first time (Render needs a minimal amount of Composition)

Centralized system

All the information to be able to render the whole movie will be in the same place (with it following backup).

The artists working in-house at San Luis, as the ones working in other areas, will use the same repository from where will be updating the information, and uploading the progress. The Subversion system will be used to coordinate the data flow.

Subversion (SVN) that allows to mantain a clean and ordered working directory, and to track the changes made by every artist, prevents data loss and overwriting of files (two people modifing the same file, lossing the changes made by one of them). Prevents the generation of loss files, that only take space and resources.

Allows everyone to know always where is everything, without need to loss time asking. And allows the automation of tasks, like the Render Farm, the automatic generation of the base files, previews, etc.

Non destructive system

The progression over a Model or Shot is lineal, one step will follow the previous in a sequencial way, but in any time you will be able to go back to the previous step without data loss, and without reasembling the files again. For example, if in the middle of Lighting process we go back to Animation, you can go forward again to Lighting with no loss on the enhasements.

Keep it Simple

Only one file will be used for every element in the movie, SVN will be in charge of versioning.
The files:

  • Camilo_040.blend
  • Camilo_040_final.blend
  • Camilo_041_fixed.blend
  • Camilo_040_ok_ok.blend
Will be just:
  • Camilo.blend
Allowing to known what is the file to use, because there is only one. We don't loss time loking on one and other trying to understand the difference. We always know where it is and which is the last version, in the first example there is no certainty about which is it.

Everything will be in the same place, will be just one folder with all the information to make the movie, from story to the render of the final cut, preventing data loss and version superposition.

For editing the same media file from story, to the final version. The media file for story will be named 0010_0010_intro.avi, the file for final render (offline) will be called also 0010_0010_intro.avi. No need to relink anything.

The files will be linked, mantanining only one copy of the information, and not multiple copies taking resources unnecessarily.

Everyone will use the only available version

Make it right the first time

The look of the image will be given on render, and only enhansed by composition (the composition will not be to set the final aspect of the image)

Tools

In order to make this some tools will be implemented:

  • SVN Graphic Interface (Blender Addon)
  • Renderfarm Graphic Interface (Blender Addon)
  • Mesh Cache export tool (Blender Addon)
  • Task managment tool (Web)
  • Render Farm setup, automatic preview render

Hide lateral panel
Blending Pipes I - Introduction
0
Blending Pipes I - Introduction
Eibriel - 2014
Eibriel - 2014

Blending Pipes I - Introduction

Blender Workflow

0

Blending Pipes II - Changes

The ideas descrived on Blending Pipes I, even when not fully applyed I think they help to make the project manageable.

I'll tell you which ideas was finally aplied, and which not, but first, a description of the work and responsibilities of a TD:

  • Network support
  • Software and Hardware support
  • Blender support
  • Servers and Render Farm support
  • Backup admin
  • Users admin
  • Pipline design and mainteinance
  • Developing Blender tools
  • Developing pipeline tools
  • Blend files and SVN mainteinance
  • Database operation
  • Render Farm operation


Svn on August 05 2013

Sometimes I resume the work of the Technical department, (or as Sony Imageworks call it "Production Services") as "make people happy", making his work simpler and easier.

In my next post I'll be describing in more detail the inner structure of the workflow, but now go back to my first presentation:

The technical aspect on the making of Kiribati is based on a minimallistic and simplistic concept. Shortening the intermediate steps to get from Storyboard to Final Render on the less steps possible.

This point rules the design of the Pipeline, I think it should be most important one on any small production.

All the information to be able to render the whole movie will be in the same place (with it following backup).

Thanks to Subversion, and the ability of blend files to be linked is posible to make this in a cheap way.

This helps in the process of Backup, and sharing files with people around the production.

The artists working in-house at San Luis, as the ones working in other areas, will use the same repository from where will be updating the information, and uploading the progress. The Subversion system will be used to coordinate the data flow.

Yes, every artist on Kiribati share the same repository, but using a system of permisions we can filter what every type of artist is allowed to read or write. For instance, an animator don't get the textures, and cant change a character file.

This way we can minimize the problems of an artists modifing something it must not, and the same time we can filter wich files an artists working from his home can download; if an animator needs to work on escene 0010, don't need to download all the scenes.

Subversion (SVN) that allows to mantain a clean and ordered working directory, and to track the changes made by every artist, prevents data loss and overwriting of files (two people modifing the same file, lossing the changes made by one of them). Prevents the generation of loss files, that only take space and resources.

SVN requires some discipline, and this helps to keep order in the files and the work. Every task start with the update of the working directory and ends with de commit (upload) of the file. Most of the time only one file is needed to shange to do the job.

Allows everyone to know always where is everything, without need to loss time asking. And allows the automation of tasks, like the Render Farm, the automatic generation of the base files, previews, etc.

Even with all the structuration and order some things are hard to find, and some things get lost. I can't imagine what happen if you leave all without control.

The automation is not always possible, subtle adjustments can be only made by a human, the base files are automatically generated (because are empty), but every artist set the blend file in his own way, making difficult to automate previews.


First Render Farm test with AFANASY, April 12 2013

The progression over a Model or Shot is lineal, one step will follow the previous in a sequencial way, but in any time you will be able to go back to the previous step without data loss, and without reasembling the files again. For example, if in the middle of Lighting process we go back to Animation, you can go forward again to Lighting with no loss on the enhasements.

Making tasks in parallel could save time, but needs extra work on planification and supervision. Thats why I forbide two people to work on the same element (shading and rigging of a character, lighting and camera adjustments of a shot). But since some pieces of an element need to be splited, some parallel work is done, not without risks and problems. For example, since the characters animation is cached we have a file with the rigged object, and other file with just the mesh and the shading; then a shader and a rigger can work at the same time. But if the rigger change the mesh the UVs are lost, or if the shader change the mesh the cache stop working. You need to be careful.

Only one file will be used for every element in the movie, SVN will be in charge of versioning [...] allowing to known what is the file to use, because there is only one. We don't loss time loking on one and other trying to understand the difference. We always know where it is and which is the last version, in the first example there is no certainty about which is it.

SVN is a versioning tool, there is not need to make your own versioning adding the date or a number to a file.

Even so, some artists have their own versions, or the Blender itself save .blend1, .blend2 files. But on the SVN only one file is saved.

Some times you need a special version of a character for compatibility in a special shot, or scene, in that case you can have a second file with a modified copy of the object.

For example:

  • house.blend
  • house_special_door.blend

For editing the same media file from story, to the final version. The media file for story will be named 0010_0010_intro.avi, the file for final render (offline) will be called also 0010_0010_intro.avi. No need to relink anything.

To be able to compare diferent stages of a shot we have different video files: "0010_010_intro_layout.mov", "0010_010_intro_animation.mov", "0010_010_intro_render.mov"

Also, we are editing with Premiere, which have the weird obsession on write on the media files and that drive the SVN crazy; so we have a copy of the previews outside the SVN for editing.

About the .blend files, we have only one from layout to animation, but it split later for compatibility with VRay and to work on background characters and maintain the file small.

  • house.blend (Layout - Animation, and in parallel to Lighting this file is used to make camera corrections and skin fixing)
  • house_extras.blend (A copy for background characters animators)
  • house_vray.blend (A copy for lighters; including background characters)

The files will be linked, mantanining only one copy of the information, and not multiple copies taking resources unnecessarily.

Absolutely. Thats why all the files needed to render the movie combined (without cache files) have a size of just 100Gb (most probably this will double to the end of the movie due to the duplication of files for lighting)

Linking files works great on Blender, with a couple of limitations and tricks, but works!

The look of the image will be given on render, and only enhansed by composition (the composition will not be to set the final aspect of the image)

Even while the movie will be rendered on VRay, and an extra work will be necesary for composition, this remains true.

Compositing is only to enhanse the colors, but nothing else.

In order to make this some tools will be implemented:

SVN Graphic Interface (Blender Addon)

No, Is hard to make an interface to SVN, it have a lot of different bahaviours, errors and warnings. It is a massive task, and could generate more problems. It was not implemented.

Renderfarm Graphic Interface (Blender Addon)

Yes, we have some buttons to send the shot to the renderfarm to caching, exporting to vray (left or right eye), and rendering.

Mesh Cache export tool (Blender Addon)

Yes, Oscurart help us with this task. I modify the .pc2 exporter to make it work with linked objects, and to reimport the base mesh and automatically add the Mesh Cache modifier and set up it.

Task managment tool (Web)

We use the Open Source tool "TACTIC", is great, we use it a lot at the begining, but with the pass of time we almos reduce its functionality to a simple database.

Well, this is an overview about how the different aspects of the "Production Services" planing develop in time.

Hide lateral panel
Blending Pipes II - Changes
0
Blending Pipes II - Changes
Eibriel - 2014
Eibriel - 2014

Blending Pipes II - Changes

Blender Workflow

0

Distributed Pipes I - Idea

A distributed pipeline for Open Movies (or not), where is not central repository, but every node acts as P2P client/server for the projects (even render nodes). Sounds good.

You can add as much artists you want, specifying their speciality (animator, shader, etc.), it will receive only the data needed for hi/her work, avoiding overload.


New aproach

Unless with centralized systems as SVN or Git, you will be able to manage Editing Media and Raw Textures (.xcf, .svg files) more efficiently.

On the Web GUI you can track tasks, make reviews, briefs, upload updates, comment.

Open Source, and Multiplatform, of course (except mobile, for now). Easy to configure, secure, cheap, no single point of fail.

The only bad: It's just an idea right now. (Simple and Powerfull)

I've tested a similar approach with my project "The Amorzorzores" and it looks promising, I'll be conceptualizing some spects so anyone can test it :)

Bests!

Hide lateral panel
Distributed Pipes I - Idea
0
Distributed Pipes I - Idea
Eibriel - 2014
Eibriel - 2014

Distributed Pipes I - Idea

Distributed Blender Workflow

0