Om kunstig intelligens, film og fantasi

31 august, 2025 er det 70 år siden begrepet «artificial intelligence» (AI) — eller kunstig intelligens (KI) på norsk — først dukket opp i offentligheten. Siden den tid har begrepet vært kjent og omdiskutert i forskningsmiljøer som har jobbet med å utvikle KI, men det var først når OpenAI lanserte DALL-E 2 i januar 2022 og ChatGPT i november samme år at offentlighetens oppmerksomhet for alvor vendte seg mot KI. Spesielt ChatGPT har satt fyr på diskusjoner om KI, og for mange er ChatGPT og tilsvarende språkmodeller synonyme med KI.

I filmbransjen er vi godt kjent med streikene som preget Hollywood i 2023. Der var bruk at KI i forskjellige deler av produksjon et sentralt tema. Selv om WGA kunne rapportere de har innkassert en sier der, og at produsenters anledninger til å erstatte manusforfattere med KI vil bli sterkt begrenset forteller mange i bransjen at bruk av KI er mer utstrakt enn de fleste vil innrømme.

Sanser, fantasi og kognisjon

I film — og kunst forøvrig — snakker vi ofte om det sanselige. Verket berører oss på måter som ikke er begrenset til det intellektuelle men heller taler til sansene våre og til det emosjonelle. Dette gjelder ofte også for de som skaper kunstverk; en kunstner tar i bruk sanser, intuisjoner og det emosjonelle for å skape. Derfor er jeg, i mitt arbeid med å utvikle en dypere forståelse for hvordan vi utdanner kunstnere, opptatt av en utvidet forståelse av vår kognisjon og kognitiv utvikling. Denne utvidete forståelsen omfatter det som de siste drøyt 30 årene er blitt beskrevet som “4E Cognition” i kognitiv vitenskap, psykologien og filosofien, hvor utgangspunktet er at vår kognisjon og sinn ikke er begrenset til hjernen men omfatter hele kroppen, sansene og verden rundt oss.

Dette betyr, blant annet, at det vi sanser og det kroppen opplever er vel så viktig for vår kognitive utvikling som det vi tradisjonelt fokuserer på, hjernen og intellektet. Videre forteller 4E kognisjonen at våre kognitive evner og kapasitet formes også av våre omgivelser og verktøy og måten vi tolker verden og vår plass i den som skapende mennesker vil påvirkes av de verktøyene vi bruker, de menneskene vi omgis av og det samfunnet vi lever i.

En beslektet teori handler om “perceptual learning“, som omfatter varige endringer i menneskers sanser, eller persepsjon, som resultat av øvelse eller erfaring. Et enkelt eksempel som ofte brukes av de som skriver om denne teorien er at en sommelier ikke bare tolker smaken av vinen på en mer avansert måte enn resten av oss, men gjennom gjentatt bruk av smaks- og luktesans vil utvikle disse sansene til å oppfatte andre ting enn det du og jeg kan. Overført til filmens verden kan man si at en fotograf vil utvikle synet sitt til å se og oppfatte ting som det gjennomsnittlige kinopublikum ikke oppfatter og en lydperson vil utvikle hørselen til å høre på en annen måte enn resten av oss. Dette frigjør også kognitiv kapasitet til å konsentrere seg om det som er essensen av faget, som de har behov for når de skaper.

Filosofer har lenge utforsket sammenhengen mellom erfaring og fantasi. Der opplysningstidens filosofer hevdet at den menneskelige fantasi er ubegrenset viser nyere undersøkelser at vår fantasi begrenses av både samfunnet rundt oss og av egen erfaring. Fantasi kommer ikke fra intet. For å kunne utvikle fantasien må vi både ha et erfaringsgrunnlag å bygge på og må arbeide for å utvikle våre evner til å bruke et vi erfarer og oppfatter som grunnlag for fantasiens skapelser. Nylig har filosofen Amy Kind (blant annet) videreutviklet dette i et bokkapittel publisert sommer 2024.

Felles for 4E kognisjon, perseptuell læring og fantasi er vekting av menneskelig, personlig erfaring. Læring, utvikling, forståelse, refleksjon, følelse, opplevelse er alle ting som krever et erfaringsgrunnlag, hvor individet på en eller annen måte har kommet i kontakt med eksterne omgivelser og bearbeidet den kontakten.

Men hva har alt dette med KI å gjøre?

For å begynne å forstå det må jeg først spesifisere hva slags KI det er snakk om. Begrepet “kunstig intelligens” brukes populært for å omtale en rekke forskjellige verktøy. Felles for dem er de bruker avanserte algoritmer, stor datakraft og maskinlæring for å utføre spesifiserte oppgaver. I filmverden har leverandører som Avid og Adobe integrert KI i programmene de lager. Flere selskap, som, for eksempel, Colourlab, har også begynt å lage KI-verktøy for grading. Det finnes mange flere eksempler som omfatter de fleste områder av filmproduksjon.

Det de fleste mener når de snakker om KI er generativ kunstig intelligens, eksemplifisert av verktøy som ChatGPT, Dall-E, MidJourney, Suno, Sora, med flere, hvor brukeren skriver en “prompt” som tolkes av KI og gir et resultat bestående av tekst, lyd eller bilde, avhengig av hvilken modell som brukes. Spesielt ChatGPT (og hovedkonkurrentene Anthropics Claude, Googles Gemini, Microsofts Copilot, Metas Llama og, i norske offentlige virksomheter, Sikt KI-chat — heretter når jeg skriver “ChatGPT” mener jeg alle slike språkmodeller) har fått mest oppmerksomhet, og de brukes til å skrive tekster, lage oppsummeringer av eksisterende tekster, idéutvikle i skriveprosesser, og mer.

Mange i filmen, også på Filmskolen, vedgår å ha tatt i bruk ChatGPT i idéfasen av en skriveprosess, hvor den kan fungere som en virtuell sparringpartner og kanskje også som en enkel redaktør i faser av skriveprosessen. Internasjonalt er det også et kjent fenomen at mange bruker ChatGPT på nettopp denne måten.

Juli 2024 kom en studie som undersøkte sammenhengen mellom bruk av ChatGPT i skriveprosesser og kreativitet. Dette var ikke den første studien, men det var den første som undersøkte effekten ChatGPT har på variasjonen i fortellingene som skapes når flere bruker verktøyet.

Og hva viser denne studien? Jo, den viser at KI kan bidra til økt kreativitet og produktivitet for en enkeltperson, og gi en støtte i en skriveprosess hvor man sitter fast eller sliter med kreativiteten. Men — og i mine øyne er dette et stort men — det finnes en bieffekt: når flere bruker KI på denne måten blir den samlede variasjonen i fortellingene mindre. Altså, bruk av KI homogeniserer fortellingene som skapes.

Her kommer vi til en annen egenskap ved ChatGPT. Selv om språkmodellene er trent opp på en enorm mengde tekst er de i bunn og grunn statistiske modeller. Selv om KI kan skape tilsynelatende original tekst har den ingen forståelse for innholdet i det den produserer. Datagrunnlaget har gitt den mulighet til å analysere hvordan tekster bygges opp rent statistisk, og den bruker dette til å generere tekst. I tillegg har programmererne skapt en del sperrer som skal forhindre bestemte typer skadelig innhold. Videre er de store språkmodellene i stor grad trent opp på engelsk-språklig materiale med de kulturelle normene det innebærer.

Resultatet av dette er der den enkelte forfatter kan finne idéer de kanskje selv ikke ville kommet på gjennom bruk av KI, vil alle forfattere som bruker det finne idéer som ligner på hverandre gjennom kombinasjonen av statistisk sannsynlighet og kulturelle normer. Og vi får mindre mangfold i fortellingene som skapes.

Det vil også kunne ha en negativ effekt på den enkelte forfatter over tid. Som jeg skrev over, er vår kognitive utvikling, våre sanser og vår fantasi og kreativitet avhengig av personlig erfaring og bearbeiding av denne erfaringen for å utvikles og vedlikeholdes. Jo mer vi bruker KI, jo mer avhengig blir vi av den.

Språkmodeller har også en annen begrensning: den forstår ikke tekst. KIs evne til å generere tekst, og menneskets evne til å forstå tekst er tydelig illustrert i dette eksempelet, som testet ChatGPTs evne til å lage oppsummeringer av lange, komplekse tekster. Kort fortalt viser testene at KI generer en forkortet tekst basert på hyppigheten av forskjellige fraser og setninger (altså, statistikk), men den generer ikke en oppsummering basert på det vesentlige innholdet i teksten. Videre er oppsummeringene påvirket av kontekst — altså din nylige historikk med akkurat denne chatboten.

Denne effekten kan ha store konsekvenser. I eksempelet lagt frem er resultatet at KI-generte oppsummeringer av faglige tekster misvisende og kan, i verste fall, presentere en annen konklusjon enn den som er i teksten. For fiksjon er det også lignende problematikk, hvor essensen i fortellingen, samt vesentlig informasjon om karakter og handling kan forsvinne i en oppsummering basert på KI.

Etterhvert som generativ video har kommet i 2024 begynner vi å se en lignende trend i hva som kommer fra de verktøyene. OpenAI lanserte Sora til stor oppmerksomhet i mars, og juni lanserte Runway sin modell som heter Gen-3 Alpha. Tester av Runways modell viser noe av det samme vi så med Sora: den skaper lett surrealistiske og til dels artige videoer, men har store problemer med konsistens og bevegelse, blant annet. Samtidig er det det også noe annet som kommer frem: verken Sora eller Gen-3 Alpha er i stand til å skape noe nytt. De lager en form for collage av eksisterende elementer, men det visuelle uttrykket er konvensjonelt.

Former KI mennesker i sitt eget image?

En sentral del av vår erfaring baseres på hvilke verktøy vi bruker for bestemte oppgaver. Utsagnet “hvis alt man har er en hammer vil alt se ut som en spiker” sier mye om oss, da det viser hvordan vår oppfatning av verden formes av den måten, og med hvilke verktøy, vi møter verden. Når vi bruker KI vil det ikke bare være et verktøy for å utføre en oppgave, men det blir en del av vår kognitive utvikling og erfaring — vi formes av KI samtidig som vi bruker KI. Jo mer vi bruker et bestemt verktøy, jo større del av vår kognitive virkelighet vil dette verktøyet bli og jo mer avhengig av det verktøyet blir vi.

Kunstnere arbeider ikke i et vakuum. Vi formes av våre erfaringer, våre inntrykk, følelser og emosjonelle reaksjoner til det vi har erfart og det vi har møtt med sansene våre. En rik og nyskapende fantasi avhenger av nye og varierte inntrykk som bearbeides mens vi skaper. Som Kind viser i sin tekst jeg referer til over vil begrensninger av vårt erfaringsgrunnlag også føre til en innsnevring av fantasiens muligheter til å skape nye og grensesprengende verk. Der KI kan hjelpe den enkelte finne inspirasjon i enkeltsituasjoner vil KI og andre algoritmestyrte verktøy også gi en innsnevring av erfaringsgrunnlaget og kan dermed føre til at det man skaper er mer konvensjonelt og likt det andre skaper.

I januar 2024 organiserte jeg, sammen med kolleger fra Høgskolen i Innlandet, et seminar om KI og kunst. Blant innlederne var Tatjana Sampojan, som kunne fortelle at i hennes virker som manuskonsulent leser hun hundrevis, om ikke tusenvis av manus hvert år. Veldig mye av det hun leser er middelmådig, og i KI-sammenheng dukker et opplagt spørsmål opp: når en KI chatbot kan produsere middelmådig tekst raskere og mer effektivt enn en profesjonell manusforfatter, hvordan skal mennesket stå frem i konkurranse med maskinen?

Hennes svar? Fire. Altså, lidenskap eller glød — det som gjør oss til levende, følende, sansende, fantasifulle skapninger og er de egenskapene som former hvordan vi bruker språk og intellektet. Der KI gir oss det kjente, det forutsigbare og det statistisk sannsynlige må vi anvende hele vårt register for å skape og fortelle meningsfylte historier.

Den 22. juli 2024 kunne vi lese en kronikk i The Guardian hvor den tyske forfatter Daniel Kehlmann innleder med å skrive,

A short while ago, a screenwriter friend from Los Angeles called me. “I have three years left,” he said. “Maybe five if I’m lucky.” He had been allowed to test a screenplay AI still in development. He described a miniseries: main characters, plot and atmosphere – and a few minutes later, there they were, all the episodes, written and ready for filming. Then he asked the AI for improvement suggestions on its own series, and to his astonishment, they were great – smart, targeted, witty and creative. The AI completely overhauled the ending of one episode, and with those changes the whole thing was really good. He paused for a moment, then repeated that he had three years left before he would have to find a new job.

Her ligger et stort ansvar på produsenter og alle som finansierer film og tv. KI er billigere og raskere enn mennesker, enten det handler om tekst eller andre elementer. Effekten er foreløpig mer synlig i spillbransjen, hvor KI har hatt en stor effekt på antall mennesker som arbeider i bransjen.

At det enkelte produksjonsselskap tar i bruk verktøy som bringer ned kostnadene når de skaper kunst- og kulturuttrykk som er kostbare og ressurskrevende er forståelig fra et økonomisk perspektiv. Men hvis det blir normen blir vi alle fattigere.

X

…as in 10. Not the other thing.

So, I’ve installed ActivityPub and Friends today, and am hopeful I can actually figure out how they work. If I tag myself — @fgraver@hcommons.social — will I see myself? Inquiring minds want to know.

Life seems to keep happening, and I have not had the chance to read and think about what I had hoped to think about. As it happens, life has kept me away from work so far, although I’m going back on Thursday.

This is not a blog post

Yeah, not into it today. For…reasons. So here is a photo I took this morning of the winter wonderland that is my back yard.

Not in the photo is the temperature — it was a brisk -22 (Celcius).

A view out my kitchen window. The porch, the yard, and the neigboring houses are all covered in snow. The sun can just be glimpsed on the left and the sky is a brilliant blue.

So. It’s 2024

Everyone needs to start somewhere, or sometime. On occasion, one might need to start several times before the thing becomes a habit.

Every time I’ve tried blogging, it’s fizzled out pretty quickly, often due to the pressure I put on myself to do it well. The weight of unrealistic expectations, or something like that.

Well. Fuck pressure. Fuck expectations.

I’m going to try just writing. Writing about whatever comes to mind, writing just for the simple reason that I want to get into the habit of writing. That means some of these posts are going to be god-awful, and I’ll have to train myself to live with that — and maybe, just maybe, as time goes on, some of them might be good. We’ll see.

There will be a mix. Sometimes politics, sometimes culture, sometimes philosophy, sometimes pedagogy. And sometimes just a random rant.

And there will be things to write about, as time goes on. I am, and have always been, one who pays attention to what is going on in the world.

I’m concerned about hypocrisy, such as the abject cowardice and hypocrisy shown by the Western world when our governments and thought leaders condemn Russia for their invasion of Ukraina but fail to do the same when Israel invades Gaza and slaughters Palestinian civilians by the thousands.

I am concerned by the rise of the militant and populistic far right in many countries, and often think about the roots of this rise, and how it’s connected to the technocratic centre which arose through the 80s and replaced the democratic control have over their lives with an increasingly regulated market economy where the rules are inflexible for individuals and are rigged to ensure global capital can operate freely. This comes to the fore in the phenomenon of “enshittification” as first identified by Cory Doctorow — a phenomenon I believe applies to many areas of the economy and not just online platforms. Here in Norway, a recent example is the privatization of the railroads, which leads to a worse experience for travellers, a massive increase in high-salaried directors of the many companies and subcompanies that now operate in a field where there previously was just one, state-owned company.

I’m concerned about the fragmentation of and in-fighting between the various groups that make up the left. I consider myself a socialist, albeit one with strong leanings towards the anarchist philosophies of Emma Goldman and Murray Bookchin. As such, I believe that social change and social justice must come as a result of collective organisation and action, and liberalistic philosophies that focus on the individual can become far too self-centred and thus antithetical to a struggle for emancipation and justice. Far too many fights in the so-called “culture wars” seem to be between those who favour collective solidarity and those who favour individual emancipation.

I’ll also be exploring pedagogical philosophies this year. I’m reading theories of creativity and seeing them in relation to the emergence of 4E cognition. I’ll be attempting to incorporate these into my research on fine arts pedagogy and seeing how all this relates to teaching and artistic research in a collective art form like film. It all begins with Dewey…

I think that’s enough for now. I’m sure other, random distractions will pop up (like sports, on occasion) and perhaps I’ll sometimes just jot down something about my dog.

Why resist?

This piece pretty well sums up why I feel a need to call to action in the wake of Trump’s election. If we accept what’s happened and meekly watch what happens next, we will be complicit in the inevitable disaster that overtakes the world.

We need to lift voices – both our own and those of others – that stand firm against ignorance and totalitarianism and call for solidarity and resistance.

The storm is coming in Europe, both in the guise of right-wing populists of various stripes but also in the guise of appeasers in positions of power who talk about “moderation” and “cooperation” as they bend their necks to the despotic forces on the rise. Both must be stopped.

The alternative is simply too dire.

Acceptance? Hell, no!

Reconciliation? Acceptance?

No.

Somehow a bigoted, racist, misogynist asshole has managed to position himself as the voice of the disenfranchised and rise to perhaps the most powerful position in the world. In Europe, his ideological bedmates are doing the same and will attempt to replicate his success.

We cannot allow this.

We must not allow this.

It cannot be that we allow these people to stand as the voice of those who are excluded and oppressed by global capital and their lapdog governments.

It cannot be racist libertarians are allowed to set the agenda for public discourse.

It cannot be that those angry at seeing their privelige threatned can continue to exercise their privelige at the expense of others.

We need to take back the fight, to once again turn the struggle against capitalism into a struggle for humanity, for emancipation, for solidarity.

Sadness is called for. Shock is called for. Anger is called for.

Followed by a cold determination and hard work.

Kjære Bjørnar Moxnes

I Klassekampen varser Rødts leder Bjørnar Moxnes en fornyelse av partiet. Dette er en nyhet som jeg, og sikkert andre på venstresiden av norsk politikk, hilser velkommen.

I den anledning har jeg, helt uoppfordret og kanskje uønsket, lyst til å komme med noen råd, i en ganske tilfeldig rekkefølge…

  1. Vær tydelig på at partiet representerer et anti-kapitalistisk prosjekt. Venstesiden, både i Norge og resten av den vestlige verden, har bommet grovt på dette etter finanskrisen, og har i stedet overlatt premissene for diskusjonen til kapitalistene. Dette må endres.

  2. Ikke vær redd for fra alle etter evne, til alle etter behov. Det sosialdemokratiske likhetsprosjektet er nå, og har alltid vært, dødfødt. Vi er ikke like, hverken i evner eller behov, og et rettferdig samfunn tar hensyn til dette. Og hele poenget med sosialisme er, tross alt, rettferdighet.

  3. Dette er veien det kapitalistiske samfunnet går. Ikke la folk eller medier glemme det.

  4. Folk er opptatt av individuell frihet, og i dagens vestlige land har høyresiden kuppet eierskap til dette. Samtidig vet både du og jeg at det er sosialisme som til slutt vil lede til reell frihet og ikke den illusoriske frihet kapitalismen lover. Rødt må gjenerobre frihetsbegrepet. Snakk om den frihet som kommer i et samfunn som ikke er basert på profitt og materialisme.

  5. Hvis du anklages for naivitet og utopi er svaret "og hva er galt med det?" Hvis mennesker ikke hadde turt å drømme om å skape en bedre fremtid hadde vi fremdeles sittet i huler. I dag sitter vi i en meget behagelig hule, men vi må tørre å drømme om å skape noe bedre.

  6. På verdensbasis er Norge priviligert og rikt. Vi nyter meget godt av kapitalismens goder, og vil nødig gi dem fra oss. Snakk om dette, og inviter til diskurs om hvordan vi – som er en del av problemet – kan være med på å skape løsningen. Ikke bare for oss selv men for alle.

  7. Uansett hva du gjør kommer folk til å forsøke å koble et moderne sosialistisk prosjekt med sovjet-diktaturene. Le av dem. Rist på hodet. Uttal deg om hvordan det er leit dine politiske motstandere sitter fast i holdninger fra 80-tallet.

  8. Vær prinsippfast men konstruktiv og samarbeidsvillig.

  9. "Sosialist" og "venstreradikal" er hedersbetegnelser. Bruk dem selv, høyt og ofte, så mister andre muligheten til å bruke dem som skjellsord.

Der. 9 korte råd. Ta dem for det de er verdt…

I solidaritet,
Fredrik Graver

Impotent fury

Most of you have probably caught all or some of this http://www.guardian.co.uk/world/2013/jun/06/us-tech-giants-nsa-data

Everything we write on Facebook, search on Google, say on Skype, etc. can potentially be spied on by the US government. This is not a surprise, really. We’ve slowly been moving towards the panopticon society for a while now, but it’s a rude awakening having it thrown in your face like this.

I have to admit this conjures up conflicting feelings of fury and apathy in me. Fury since this is just an unacceptable way of constructing a society. We are all being criminalized and our freedoms are being stolen. Apathy since my life, like so many others, is wrapped up in digital connections and I don’t know how to disconnect without losing much that I value and am not willing to give up.

Gramsci was absolutely right. There is a ruling hegemony that finds cohesion in the need to keep the great mass of us docile and compliant and preoccupied with other things than tearing down the system.

Jeg ble ledet til denne bloggen via en av mine medstudenter i #etmooc, en online utforskelse av utdanning, teknologi og media. (www.etmooc.org). Akkurat nå handler det om “the open movement”. Samtidig har jeg nylig leste en del av det som er skrevet om Google Glass, og hvordan det kan påvirke oss når vi plutselig vet at alt vi gjør og sier kan bli tatt opp og lagret.

Det er mange år siden jeg leste Foucault, men han virker mer relevant nå enn da jeg leste ham på 80-tallet.

Insight Out raw

I attended the 2012 Insight Out seminar at the film school in Potsdam, Germany from March 19 to 23 this year. I am dumping my “raw” notes here; they are unfiltered, unspellchecked and a little unstructured. They are useful to me – if nothing else, the act of taking notes helps me remember things – but probably not to anyone else…

Day 1
_________________________________________
Session 1
general overview of workflows and formats in digital film and home entertainment
– by Flying Eye – media management consultants; ie getting the techies and creatives to understand each other.
Background: jan. 19th Kodak files chapter 11 bankruptcy protection.
Film continues to dominate archiving, but digital acqusition taking over in production – one could say has taken over.
SDTV –> HDTV –> 4K (digital cinema)
Digital workflow allows for crossover between production and post; this is not only possible but even likely. (and desirable?)
– fewer steps digital – digital than film-digital
– DCI is still active; produces SMTE standards. last revision 26.01.2012 www.dcimovies.com
– DCI focusess on distribution & security
– DCI also gives specs for testing of distribution
Last 3 years have seen an almost exponential growth of digital and 3d screens compared to film-only
Rumour that Hollywood will stop releasing 35mm in 2013. This will have a major effect on those screens that have not converted to digital –> but what about arthouse? archiving? less industrialised territories? second run? festivals?
Issues:
– projection brightness
– higher framerates (48fps)
– heigher resolution (8k+)
(these last two have big implications for production and post pipeline)
– industry still learning to handle 3d
– film language for 3d still under development
As always, the choice of camera is chosing the right tool for the job – should not be a choice based solely on technological considerations
Interesting development also in DLSR; both Canon and Nikon have released new cameras this month. N D800 (better colour but more noise) and MKIII (less noise but more moire and false colour)
Brand new! Digital Bolex announced last week.
Acquisition formats
– resolution
– bit depth
– compression
– colour space
mm.
Field recording: disk based, flash memory (SSD, etc.) and tape based
With online storage of masters, one can move away from a “sequential” post to a collaborative one.
– also Cloud-based. Example, Hugo Cabaret moved 3PB data between 11 different collaborators in post. ref Pixomondo – teams all over the world working on the same production.
Consumer delivery: IMF – interoperable master format
– standard for mastering for all consumer delivery formats.
Frame rate: 24 vs. 25 fps –> DCI compliance requires 24 BUT the latest standards allow for 25 fps DCPs for cinematic release; but even though the standards exist, the equipment to play 25 fps is not widespread.
___________________________________________
Session 2
Broadcast Changes.
John Ive from IABM (http://theiabm.org/)
represents the broadcast equipment manufacturers
analogue –> digital –> IT/IP
– this last element is what has actually changed workflow
More delivery platforms and consumption habits are changing; leads to both creative, business and technological need to adapt.
– how to future-proof in a situation where change is accelerating?
Drivers in the consumer world
– live programming
– multichannel pay-tv
– recorded media – multi-format
– video on demand
70% of tablet owners use them while watching TV (UK)
Daily viewing hours still increasing
From tape to file – file = video + audio + metadata
Analogue and digital are still linear / sequetial workflows –> IT/IP changing all that
IT is faster than real time and paralell
“ingest once, use anywhere”
But why are we struggling?
– have to unlearn ingrained workflows
– exponential rise in content
– formats!
– cost of investments and upgrades
– too many things changing at the same time
Metadata – provides the context necessary to make sense of data.
– there is no “one size fits all” as different organisatons need different metadata.
Workflow
key in IT-workflow becomes media asset management.
Existing cloud services – worth checking out to see the (possible) future of media asset management.
Security is an issue in cloud storage
– local laws where the servers are; which agencies can access my files?
– leakage and hacking?
_______________________________________________
Session 3
Peter Adams – editor of Anonymous
editing and VFX
– shot on Alexa
– communication between editing, VFX and production was key in both prep and production
– got technical files from the camera. DP predefined different luts that were applied to the dailies and a form of pre-grading
–> gave DP a chance to pre-grade, and the looks were changed along the way.  Eliminated dailies colour grading and predefined som set-ups for the colour grading
– raw material recorded to hard drives, copied at the end of each day by a quality controll dept. – replaces the lab
– this dept. produced timed and synced dailies, already logged and organised.
Editor and director watched dailies at the end of each shooting day
– Started editing while the shoot was going on.
– often involved sound and vfx while cutting scenes in order to get the full effect
VFX and editing started working and testing together about a week before production started.
– technical compatibility as important as collaboration
Good communication also key to ensuring efficiency; avoid wasted labour
– back and forth collaboration allowed adjusting plans to ensure maksimum efficiency
By cutting along the way, the editor had a cut ready for a (internal) screening about 2 weeks after principal photography was finished.
– worked another 2 weeks on a directors cut
– another couple of weeks, then another screening
Editing moved to LA, but VFX still in Germany.
– an assistant stayed in Babelsburg, but editor and main assistant moved.
– conference calls with VFX every two days.
– used a watch folder (dropbox) and a player called cinesync – which allows you to draw on images and the drawings are synced real time between LA and Germany
– moved editing to London in so the director could do ADR with the actors
Recreating historical London from before the fire: combination of VFX and production design – CGI and sets, set extentions, etc. This required extensive previsualisation between those two depts. – but editing was not involved in animatics, etc.
______________________________________________
Session 4
s3D Now! – research project
On the effect of 3D on storytelling, production and sound, in addition to testing workflows
3 main research questions
– optical phenomena in 3d and their limits
– staffing on 3d productions
– sound in 3d films; how is it different from 2d
Project entails a short film, a behind the scenes doc and a book
Sound for 3D – there is no litterature right now
– how does 3D effect set sound and post sound?
some challenges
– clocking with many devices
– stereographers equipment is noisy
– same is true of data wrangler
Shooting
– more wide shots, longer shots, fewer ots. –> difficult to get mics close
Post
– need 3d image for sound post in order to miks to the correct space
– increased detail level in sound design
– new approaches for surround channels since the audience is inside the space and sound must reinforce this
– may need different playback systems – eg. 7.1
Sound concept
– must remember the room / space is a stylistic device ie. a storytelling device
– directors must learn to include more departments
MORE PREP.
3D Alliance Karlsruhe
investigating storytelling in 3D
forms alliances between art and science (esp. mathematics)
Beyond – 3d filmfestival
* 3D symposium 22-24 june. *
Working towards directing attention from technical aspects to creative aspects
big challenges: no standards and lack of educated people.
3D requires a new dramaturgy; one that uses space actively
– both in terms of staging and emotionally
Requires new rhythms in editing
“in 3d, the post people become painters”
___________________________________________________
DAY 2
Session 1: The Law of the Internet with Prof. Jan Bernd Nordemann
Intellectual Property rights: remember everyone who has had creative input in the film — editor, cinematographer, actors, composers, etc. — owns the intellectual property rights to their work. The Producer needs to remember to acquire all these rights for distribution.
– in particular internet and “all unknown media” rights
– need explicit agreements for “non film use” –> for example merchandising, books, etc.; also prequels, sequels, remakes, etc.
Old programmes – anything before 1995 – are problematic for internet distribution
– especially from before 1965
Why? Before 1965 no mention of unknown media in contracts. –> rights remained with the creator/author
In 2008 German law granted distribution rights to producers for content created 1966-late 70s for what were “unknown media” in that time period. ie. video, internet.
– The authors can make a separate remuneration claim to a governing body.
In US the situation is different under “work for hire” provisions –> producers retain more distribution rights
In the case of full funding by a broadcaster, the internet and unknown media distribution rights may be with the broadcaster. At the very least, the broadcaster retains the “7-day catch up” distribution.
– VOD a separate issue.
No one “right” on the net. Many different forms and rights of distribution on the net.
– EST – electronic sell through (purchase)
– VOD – on demand viewing by time-limited download, but not sale.
– Streaming – on demand viewing, but no downloaded copy for viewing
Remuneration
– free
– free with advertising
– pay-per-view
– subscription
– etc., etc.
Geographically limitet rights – based on geolocation of IPs
– there is a question about the EU; must rights be whole EU or can there be rights specified for individual states
Music rights are the most complicated thing, in part because of national composers & musicians rights organisations. In some cases, rights must be negotiated with individual performers and composers –> this is a major hurdle for film distribution.
Another aspect: live streaming of TV.
Creative Commons. Has become part of the copyright system, but is free of remuneration. –> since it is noncommercial there is a limit on the kinds of content that will be CC licensed.
Platforms have to adhere to national laws for all territories in which they are available.
– example: youth protection (age limits) could determine at what time you can stream certain content in certain territories
Piracy. Still unresolved. Copyright vs. individual freedom.
_________________________________________
Session 2: Production Designer Sebastian Krawinkel about Anonymous
Every scene in Anonymous was designed, and built either physically or digitally.
Used a modular system – elements that could be moved and reused in different ways.
Original idea was to shoot plates in England, and use them in CGI. In actual fact only one plate was used.
Made models which were shot, then manipulated in VFX to create London
Art dept. researched and built large parts of 16th century London – Art. dept. made many 3D models, then handed these to VFX for completion.
Krawinkel had 12 weeks prep for research.
Based sets and 3d designs on contemporary paitings of London. Houses based on existing houses that survive from that era.
Extensive previs, both 2d and 3d
Close working relationship between Art Dept. and VFX in prep.; sharing of concepts and files; planning of physical and digital elements
In the interiors, modules were built and reused in many sets; simply fleshed out with some unique elements in order to distract the eye from the reuse of elements,
Primary interior lighting from candles; used “candle wranglers” and special 3-wicked candles. (Chose to shoot Alexa because the candlelight caused noise in in the footage from the Red camera that was tested)
In certain situations, the modular sets were combined with CGI and digital sets
Used 5 different sets, so that while one was being filmed, the other 5 were in various states of preparation.
Modular system works when architecture is geometrical, the director is very visual and can see possibilities, and the AD can make a schedule that works.
__________________________________________________
Session 3 – 3D storytelling and workflow planning
Part 1 with Ludger Pfanz
3d has a history all the way back to Lumiere and Meliers
Digital 3d allows perfect sync in projection
Theses
– 3d starts with the script
– from “frame” to “stage”
– space has meaning
– depth dramaturgy
How to organise space for 3D
– Topos: space –> space has potential and dynamism
“What is not in the script cannot be staged” – scriptwriters need to learn to write for 3d space.
– for example, in Coraline the day world is flat, while the night world has a lot of depth
Can layer an image to have more going on different levels –> mise en scene
Alex Weimer & co. – “Lost Place” a case study in 3D filmmaking
Mystery thriller, based on real backstory of US cold war experiments.
How to make the 3D count?
Things that lend themselves to 3d
– ensemble cast
– limited number of locations, which are explored by the characters
Depth chart: an analysis of the script to decide which scenes need more or less depth, where parallax is in front or behind the screen
Can increase Vertigo-effect by increasing inter-axial distance between cameras (for example) –> puts a strain on the viewer
Production considerations
– stereo rigs can weigh in excess of 40 kgs.
– changing rigs can take several hours of calibration
– stereographer becomes a key crew member; important to give them the time and space to work
– special rig for steadicam – down to approx. 25 kgs.
– stereoscopic mirror rig eats about 1 stop
Editors observations.
kept each eye separate – even on separate hard drives
Used Avid media composer 6 – latest version has 3d capability; allows linking two picture and one sound clip to create a stereoscopic clip.
What about pacing? – the pace of the editing and choice of footage to use determined by the action and shots exactly as with 2d
Cut in 2d, then view sequence in 3d.
Some differences
– backgrounds are a bigger distraction in 3d – even if out of fokus. Need to be aware of this
– tracking shots can, if necessary, be held longer in 3d.
– lens flare and reflection is a bigger problem in 3d.
VFX and post production; on set data wrangling
– data integrity key; backups and more backups.
– DIT followed file naming convention agreed upon in prep.
VFX did on-set check and screenings of footage; used After Effects since the editing workflow had not yet been established
Involved in prep in order to avoid overwork in post 😉
gathered textures, etc. on location
built websites, graphics, phone apps, etc. etc. in advance so they were filmed “live” during production – again to avoid work in post.
Have not yet decided on software package, but considering Fusion rather than Nuke.
Big challenge creating effects that also have to have depth.
_______________________________________________
DAY 3
Session 1 – Case Study ARTE – broadcast workflow in the transition from tape to file
ARTE is a grouping of public television networks; French and German cooperation, started in 1992
Head office in Strasbourg
Tapeless: started with SD tapeless in 2001; went live in 2005
Introduced HD in 2008
APIOS: database for managing broadcast assets; uses metadata to automate programming, archiving, etc.
– also manages audience number, etc.
– this system is available for all now, can also be used for archiving, vod, etc.
HD-tapeless: decision to go tapeless made in 2008
BCE won the contract to effect the changeover to tapeless
APIOS is based on Oracle and on Windows-only.
Going tapeless has much simplified by removing steps, need for synchronisations
(MAM = media asset management: key feature of tapeless system)
In a changeover like this, change management — training & listening — are key
To any such system, metadata is key –> it is crucial that this is entered and managed correctly.
_________________________________________
DAY 4

Session 1: Case study Melancholia: Pan-european co-production with Peter Hjorth
check http://www.gearless/MEL%20VFX%20featurette%201stcut.mov

NB: Peter works exactly the way we want to teach Digital Visual Design.
Melancholia had collaborators from Denmark, Sweden and Germany on the post production effects.
Peter worked as a “VFX Director” – closely with Director, DoP and PD throughout the production cycle
Partners in VFX included:
– Pixomondo in Germany
– Film Gate from Sweden
Peter and his VFX producer Karen were hired at the script stage
– made some models, working with the scientists who provided a large amount of the background reseach for the film.
– first VFX breakdown and budget already two months before preproduction started; worked closely with the films producer to find a solution that works with the total budget

Storyboard
– Peter worked with Lars, DP, PD and 1st AD
– final film very close to storyboard
Developed the colour and look of the Planet Melancholia; worked with Gaffer and DP so they could find a gel they used to give the effect of the light from Mel. shining through windows, etc.
Predefining what is needed; working with production to ensure what is needed is filmed.
Karen did cost estimates to see what is best done on set, what is best done in post.
The general trade-off is big, wide shots better as post VFX, while smaller, closer shots better as on-set SFX
Involved in planning 2nd unit shooting – ended up saving time and money
Made a production plan for FX shots

Financing
breakdown based on final script
gave all studios same breakdown, storyboards in order to get best possible bids.
shots were cut all the way until the end in order to make the budget work, or in some cases redistribute resources
Economy not the only factor in chosing post studios
– artistic style and visual understanding
– experience and capacity
– key competences
– personal contact
– location (travel time for Peter and Karen) –> this was a factor in chosing German Pixomondo over a Canadian competitor.
Tight schedule: shooting finished in September; film had to be ready for pre-Cannes screening in March
Budget har to consider shooting costs, personnel, construction, post hardware, travel, file transfers, etc. etc.
4-country co-production; there are national demands for where budget is spent – VFX also has to help by spending in those regions… (plus a large list of other investors)
— Worth noting that Film i Väst (Göteborg & Trollhättan) is a major booster of filming and post in that region —

The Shoot
Peter on set all days when vfx-shots planned
Input in how actors responded to the Melancholia planet in the sky
participated in shooting reference elements, plates, inserts, spinter unit with sfx and art dept. (most high speed); 2nd unit of varying sizes (inkl. model shoot), special elements (in this case, Northern Lights on Iceland), pickups (greenscreen)

Editing
Important to keep track of the shots used, that the vfx plan is still being followed; see if new shots come up, etc. –> important to follow up that the total VFX shots stays within the budget.
Peter takes job of VFX-editior: does rough versions of vfx shots once there is a “real” cut of the film available
Make vfx-demos for the post studios involved, allow them to start sketching their versions while waiting for the final version of the picture
Post studios provided temp versions for screenings

VFX
Online of film to vfx studios
briefings and presentation of shorts
VFX production schedule finalised
maintain communication and make visits, viewings — this is key to ensure what is delivered is what was wanted.
VFS shots to foley, sound editing, music
Coordination important, to ensure everyone knows who does what, when, and also movement of files, scheduling of approvals and viewings, etc.
Important also to coordinate with Producer to fit promo and trailer production into the schedule; they can really destroy a post schedule otherwise.
Peter works much like a director with the vfx-providers; is the creative leader of the team and needs to get the most out of the different artists.
– tools like Skype and Cinesync are useful, but there is no substitute for sitting and speaking in the same room.
worked with ftp –> still the easiest way to distribute files.

Grading
Peter supervises the delivery of every vfx-shot; also attends final grading
involved in creation of LUTs for dailies; ensure all vfx shots work with the LUTs
Grading requires delivery of alpha layers; mistakes in grading can ruin the effect of compositing!
Things like stars can cause major problems in downconversion for release formats — all these versions must also be checked.

This case shows how important it is for the FILM that the VFX director is involved from script to delivery.

Q&A
Alexa – shot 10-bit log with on-board compression codec.
______________________________________________
DAY 5

Session 1: Visual effects photography and 3D with Mark Weingartner
Visual effects and CGI are not the same thing: not all vfx are CG, although they may have CG elements.
– many elements, plates, miniatures, etc. are shot, then manipulated, composited in post.
Useful book: VES Handbook of visual effects
Things like Vistavision still used for f/x
For 3d models – use lidar scans instead of photos for plates.

Session 2: “101 It’s Easy” –> new company helping production companies make transition to file-based
Protection of media is key
Digital allows sharing and collaboration to a much greater extent; this increases complication
Cloud-based file management is key to collaboration and sharing; producers need a system that can be secure but flexible.

Session 3: Dolby PRM-3200 reference monitor
John Bleau
New monitor a response to the lack of a proper display in post after CRT discontinued.
– main problem is there is no longer a universal reference standard
– LCD and plasma have severe colour and black/contrast limitations.
– OLED and plasma have a short life
– DLP best colour, but poor contrast, poor blacks
Dolby monitor 40000:1 contrast, 12-bit colour. Can handle raw output from Alexa, for example.
$40,000 –
Calibration probe is another $30,000…

Session 4: Digital Gremlins! with DP Keith Partridge (films extreme locations, etc.)
or… the many deaths of digital footage…
One way to kill a camera…–> drowning
Cameras don’t like water! Adventure cinematography can get wet…
Other ways –> dropping and freezing.
Logistics of the shoot determine choice of equipment
When one is in a remote location it’s crucial to use things one is familiar with –> if something goes wrong you can quickly fix it / find a solution

Session 5: Pixomondo and Hugo Cabaret
shot on Alexas with a 3d beamsplitter rig.
extensive metadata recorded
Nuke for compositing
VFX involved in setting up the LUTs
One pitfall of 3d: stereo artifacts
– 2-stage removal process: automatic tools followed by manual cleanup
– not all stereo artifacts are in camera; some are real-world etc. occlusions and reflections –> in both cases they are visible to one eye and not the other; creates an annoying effect and need to be dealt with.
Tools used included both rotoscoping and CGI
in this case: vfx was not involved in chosing shooting format.

Session 6: Summary
SD and tape (with the exception of HDCamSR?) are gone from post; soon from production?
Codec chaos…
Film?
In broadcast today, 1080i50 is the format — but not the format of the future.
How many pixels are enough?
Chosing a camera today is becoming as much a creative choice – much as chosing a film stock (Kodak vs. Fuji) was before
Next technological advancement on display side is autostereoscopic (no glasses) –> holoscopic interesting but still a ways off
Stereoscopic 3D is here to stay, but needs to be driven by story, not money!
Filebased –> leads to demands on coordination, integration and compatibility in order to allow effective creative sharing.
– key question is how to make the technology/infrastructure invisible so the creatives can get on with their work!
Archiving: digitise tapes or lose them. Film is still supreme on the theatrical side, but what will replace it? Very important to resolve this.
In broadcast, adminstrative and production worlds are moving closer – driven in part by consolidation of databases. How close should they get?

As an aside, this was the first time I used my iPad 2 as my primary computing device for an extended period.  The laptop stayed in the hotel room while I wrote, checked mail, etc. on the iPad with my Logitech fold-up keyboard (Norwegian link). The battery lasted all day without trouble (used about 10% per hour when in use constantly – just as advertised).

css.php