Houston Choral Community "United in Song" Virtually By SYME

Stretch Your Mind Entertainment was proud to produce the “United in Song” livestream concert on behalf of five different choral groups in the Houston area. The massive scale of the project required nearly six months of planning and coordination to bring to life. The project also garnered SYME’s first media coverage in a story by the Houston Chronicle.

The December 18, 2020, livestream of the finished product, simulcast across various social media outlets for each of the ensembles and production companies, was hosted and directed by Nathan Abbott of DagNAbbott Production and Design.

Due to licensing restrictions, the complete broadcast could not be redistributed in its entirety, but each of the segments from the show in order can be found in this YouTube playlist compiled by Houston Show Choir.

In addition to producing the segments for Houston Masterworks Chorus, SYME created the grand finale of the broadcast: an enormous virtual chorus of 212 musicians from all five groups singing “Holy Night, Silent Night” arranged by Ruth Elaine Schram.

Behind the scenes

The first step is sketching out a rough gameplan with the score

I generally have very specific ideas in mind all the way at the beginning of this process. Much of what I write in the score makes it all the way to the final product.

Early sketches of various different grids for Masterworks and the finale testing out ideas for layouts

Once I knew exactly how many people I had to work with (210 singers + 2 instrumentalists), I could come up with a design for the final grid. 210 was a tricky number to work with and get everyone to fit. This was just the final design of numerous sketches with different sizes of piano/violin and rows/columns

A first take at distributing the parts. Usually, in my videos, I like to keep the grids arranged roughly in the order folks would appear on stage in sections. For this special finale, with all the groups coming together, I wanted to represent diversity by having an even distribution of men, women, parts, and even by ensemble. This was not the final layout, but it was close.

A spreadsheet with all the singers, their parts, and ensembles in hand, I began to meticulously make the assignments and necessary tweaks to the final layout to make it work the way I envisioned.

The next step is audio. All the editing and compositing is timed to match with the music, so it’s important to get a decent pass at all the singers synchronized early on. This is a screenshot of Masterworks audio with a preview clip of violin in the background to check the sync.

With the final assignments for the grid and a version of the audio everyone can be synced with, it’s time to create the grid itself. The so-called “Tutti grid” is always the hardest one to assemble of any project, so it’s always the one I start with.

In the very first conversation I had where Paulo was pitching this project to me, he mentioned he wanted to have a finale with all the singers holding candles. For a number of technical reasons, we ruled that out almost immediately. But that first conversation made me determined to find a way to have singers with candles in the video. The week of Thanksgiving, I finally had the idea for a “field” of candles that would have singers attached to them. Unsure exactly of the logistics of that, I impulse bought a bunch of candles. Once they arrived, I realized that an easier way to make it happen would be if all the candles were in a row, as if it was a solemn procession. Here I am measuring out the candles, spaced exactly 9 inches apart.

I began to have second thoughts when I looked back at how far I had to go

There are 53 candles in the procession. The gap in the middle corresponds to the brief instrumental interlude. In this picture, I actually had put the gap in the wrong place because I apparently can’t count.

I wanted the candles to advance at a uniform rate and needed specific candles to be on screen at very precise times. I needed to figure out where to have the camera so that each candle was in focus and close to center screen at its assigned time. I attached a laser rifle sight to my crane and marked on masking tape next to the candles where the laser dot needed to be for a given candle to focus.

Performing some practice runs with lights on to get the feel for the pacing. I made a copy of the audio with a metronome click track that clicked at the same rate the candles should appear. I played the track over and over to practice hitting the laser marks on each of the clicks in the audio track.

Checking the framing one last time before lighting the candles

There are no lights on in my house in this picture. This is the ambient light just from the candles. It was much brighter than I expected it would look, so I had to artificially darken the shot with color grading.

The morning after. Some of the candles spilled over and required a few hours with a putty knife to clean the wax melted to the floor.

Building the quartets by hand. I wanted to have every member of each quartet be from a different group. There are two exceptions to this rule. Simply because of the imbalance of people from each group, men and women of the same group could appear together, but I tried to minimize this. The other exception is a bit of an Easter egg for the production team. I had produced a early draft of the finale for the directors to review featuring a single quartet of Masterworks singers. They appear together in the candles as well as the center four squares of the Tutti Grid.

Not only did I want different groups represented in each quartet, I wanted to mix up all the parts so it wasn’t always the same SATB order. The quartets, as well as each of their members, were assigned and sorted by random numbers. The next step was to tie each of the singers to the specific laser marks/candles.

I wrote a Python script (one of six I wrote for this project to process audio and video files automatically) to parse the list of names and times from the spreadsheet. It would then automatically generate the code for my visual effects software to add the clips to a virtual 3D space mapped to the camera movements from the candle video.

Even though most of the nodes of the flowchart were automatically generated by the code, I did have to make manual tweaks to things which ended up forcing me to manually rewire some parts of the flow.

Some early tests to figure out the right amount of transparency, sizes of the images, and refining 3D placement.

I spent nearly a week processing the original candle video to generate tracking data. The software tried to identify points in the video it could track for a few seconds at a time to solve for the camera’s position and orientation at each instant in time. This process is not well suited to nearly four-minute single takes. This meant I had to manually cull and refine the tracking data to try to get a solution that looked like something resembling the real-life actual motion of the camera. This was the first time I got something that looked right, so I added the first quartet as a test.

From the side view, you can see the raw tracking data along the bottom. The camera clearly identified most of the candles, which you can see sticking up above the “ground” line. Because the final tracking was much less precise than the actual shoot, I ended up having to manually place all of the quartets in the 3D model.

Because the process of creating the candle quartet animation took so long, only the production team working closely with me on preparing the broadcast actually got the see the finale before it aired. All of the directors were stunned by the result. I celebrated being done with the preparation by projecting the broadcast in my garage with a distanced friend in my front yard. A significant accomplishment for all involved in making this show happen!