Computational Cultures Lab

Technical Guides
Guide 5: Machine Vision, Generative Ai And Assistants

Introduction

Research on artificial intelligence has progressed relatively slowly, beginning in the mid-20thcentury with the pioneering work of English mathematician Alan Turing, and his ‘Turing Test’ for artificial intelligence (Turing, 1950). While various models were advanced and developed over the subsequent 50 years, including the symbolist, connectionist and inference based approaches, it was not until the introduction of advanced video graphics cards in the late 1990’s that provided exponential increases of computational power and the emergence of ‘big data’ as a result of high-speed internet, smart phones and social media that the bottlenecks in AI research were overcome. In quick succession since the mid 2010’s, there have emerged a series of breakthroughs in AI which have brought us closer to artificial general intelligence (AGI). These breakthroughs include the emergence of ‘deep learning’ using convolutional neural networks whose AI architecture replicates the neuronal structure of the human brain. The development of generative adversarial networks (GANs) would lead to the emergence of ‘deepfake’ images and photo realistic video which could simulate the appearance of individuals, celebrities and even create entirely new humans. In the late 2010’s, the discovery of ‘transformer’ networks would give rise to the current generation of large language models (LLMs) and generative AI which are the basis for AI assistants like ChatGPT which are capable of human like conversation, generating written text, coding and high-level reasoning to solve mathematical and scientific problems.Since the mid-2020s these systems have continued to advance. Newer models support longer context windows, multimodal input (text, images and audio together) and more structured step-by-step reasoning. Open-source and lower-cost models have also become more widely available, making advanced language and reasoning tools accessible to a broader range of users and researchers.

ChatGPT and OpenAI

In 2018, OpenAI, the company set up by prominent Silicon Valley figures like Sam Altman, Elon Musk and Peter Thiel to conduct open-source research on AI, released their generative pre-trained transformer (GPT) model which used a decoder only LLM to generate language. Each subsequent version of GPT increased its training data set to become more precise. GPT-1 was released in 2018 and had 117million parameters and was trained on the BookCorpus dataset which contains 7000 self-published books. The company followed up in 2019 with GPT-2 containing 1.5billion parameters and trained on text from the online message board Reddit. In 2020, the company released GPT-3 which featured 175 billion parameters and was capable of writing articles, code and translating languages. It could also recognize new concepts or objects without prior training because of the immensity of its training set, a feature known as zero-shot learning. The model was trained on datasets taken from CommonCrawl based on 8 years of webcrawl data, WebText2 containing all data from all pages linked from Reddit, the Books 1 & 2 internet book archive datasets and the open-source, user generated online encyclopedia Wikipedia. In 2024, they released chatGPT-4 which is multi-modal, taking images, audio and video as well as text. It also has natural language capabilities, being able to converse with human-like emotion. By 2025–2026, reasoning-focused models and strong open-source alternatives became common. These systems often show intermediate reasoning steps and handle complex multi-step problems more reliably than earlier versions. Access methods and exact model names continue to change frequently.

Generative AI

Diffusion models are another form of generative AI which are used as the basis for well known models such as Stable Diffusion, Dalle-E and Midjourney. Diffusion takes images from a training data set, and gradually adds a type of noise to the image until the image becomes static, and then uses reverse diffusion and deep learning to denoise the image until it generates an image similar to the base image in the data set. The first step involves taking the users text prompt and relating it to datasets of images with text-based captions and descriptions which are used to convert the prompt into numbers. The second step is the decoder phase, in which a diffusion model is used to generate an image based on the keywords that link to the labeled original dataset. Diffusion models have also been used in text-to-speech applications to generate and simulate human voices. DALL-E was released by OpenAI in 2021, which could generate high resolution images from text prompts. Another popular diffusion model is Midjourney, released in 2022 which was accessible to users via their Discord server bot and generated images via their cloud servers. In 2022, Stable Diffusion was also released, an open-source latent diffusion model which allowed users to download the foundation model and run it locally on their own computers. The first generation of these generative AI platforms were mostly limited to still images and very short clips, but with the release of OpenAI’s SORA and Stable Diffusions collaboration with RunwayML, longer duration generative video has become available to users which has led to an explosion of companies offering these services. As a result of Stable Diffusion’s open-sourcing of their model, online communities on sites like Hugging Face and Civitai have also emerged for users to fine tune customized generative AI image and video models via Low Rank Adaptors (LoRAs) by training their own data sets, and add further fine-tuning capabilities to the base model via the ControlNet neural network structure.Image generation has improved in consistency, text rendering inside images, and the ability to maintain character or style across multiple outputs. Video generation has also advanced from very short clips to longer, more coherent sequences that can maintain visual continuity and camera movement. Open-source communities continue to develop fine-tuning methods (such as LoRAs and ControlNet) that allow users to customise base models with their own data.

DeepSeek

A major criticism of the current model of generative AI and LLMs pioneered by companies like OpenAI are the enormous energy resources and compute power needed to train their foundation models. Another critical dimension of this approach is that they are based on closed application programming interfaces (APIs) whose code is proprietary which means that academics cannot use them in their research since their architecture remains a nominal ‘black box’. In January of 2025, a relatively small Chinese startup named DeepSeek released a large language model capable of reasoning at a level comparable with OpenAI’s leading models, which they claimed cost 6 million US dollars to train, compared with the 100 million price tag of GPT4. In addition, the model was released free and open source so that any startup, large company or researcher could use and build customized LLM reasoning models. Subsequently, major system weakness have been revealed in Deep Seeks architecture which make it vulnerable to security leaks and hackers, however, the game changing nature of Deep Seek is still being felt in the industry.

Tools

[Table]

  • Software
  • Key features
  • Access

Chat GPT 5

  • Generative AI chatbot
  • [Key features]
    • AI assistant which can mimic human conversation
    • Ability to write and debug computer programs
    • Ability to generate text in any format eg. Essays, stories, poems etc.
    • Closed API
  • Paid with subscription

OpenAI 01

  • Generative AI chatbot
  • [Key features]
    • Designed for complex reasoning tasks
    • Can solve mathematical problems
    • Can solve science problems through ‘chains of thought’
    • Capable of writing complex code and computer programs
    • Closed API
  • Paid with subscription

Google Gemini

  • Generative AI chatbot
  • [Key features]
    • AI assistant which can mimic human conversation
    • Ability to write and debug computer programs
    • Ability to generate text in any format eg. Essays, stories, poems etc.
    • Reasoning capabilities
    • Machine vision capabilities ie. Ability to understand images and video
    • Advanced coding capabilities
    • Closed API
  • Free

Deepseek

  • Generative AI chatbot
  • [Key features]
    • Designed for complex reasoning tasks
    • Can solve mathematical problems
    • Shows ‘chain of thought’, or logical steps the system took to generate answer.
    • Capable of writing complex code and computer programs
    • Open source
  • Free

Copilot

  • Generative AI chatbot
  • [Key features]
    • AI assistant which can mimic human conversation
    • Ability to write and debug computer programs
    • Ability to generate text in any format eg. Essays, stories, poems etc.
    • Reasoning capabilities
    • Machine vision capabilities ie. Ability to understand images and video
    • Advanced coding capabilities
    • Closed API
  • Free with Microsoft 365

Claude

  • Generative AI chatbot
  • [Key features]
    • AI assistant developed for automating simple and complex computer tasks
    • Ability to write and debug computer programs
    • Reasoning capabilities
    • Advanced coding capabilities
    • Closed API
  • Chatbot available for free, more complex operation available through Pro subscrition

Llama

  • Generative AI chatbot
  • [Key features]
    • AI assistant developed for automating simple and complex computer tasks
    • Ability to write and debug computer programs
    • Reasoning capabilities
    • Advanced coding capabilities
    • Open source
  • Open* source and free

Poe

  • Platform for accessing generative AI chatbots like ChatGPT
  • [Key features]
    • Gives access to AI assistants like ChatGPT which is currently unavailable in the Hong Kong SAR and China
    • Ability to write and debug computer programs
    • Ability to generate text in any format eg. Essays, stories, poems etc.
    • Reasoning capabilities
    • Machine vision capabilities ie. Ability to understand images and video
    • Advanced coding capabilities
  • Free

Midjourney

  • Text prompt* based AI image Generator
  • [Key features]
    • Ability to generate images from text prompts
    • Able to use ‘style transfer’ to apply the style of one image to another
    • Can be used to colourize black and white images
    • Can generate new images based on a single source image provided
  • Paid with subscription

Stable Diffusion

  • Text prompt* based AI image Generator
  • [Key features]
    • Ability to generate images from text prompts
    • Able to use ‘style transfer’ to apply the style of one image to another
    • Can be used to colourize black and white images
    • Can generate new images based on a single source image provided
  • Paid with subscription

Sora

  • Text prompt* based AI video Generator
  • [Key features]
    • Ability to generate video from text prompts
    • Can be used to create video from still images
    • Can be used to extend the length of video clips
  • Paid with subscription

Pika

  • Text prompt* based AI video Generator
  • [Key features]
    • Ability to generate video from text prompts
    • Can use ‘style transfer’ to apply the style of one image or video to a new AI generated video
    • Can be used to create video from still images
    • Can be used to extend the length of video clips
    • Can generate storyboards from text prompts
  • Paid with subscription

Runway ML

  • Web based AI platform
  • [Key features]
    • Gives users the ability to train AI models without coding
    • Create custom image datasets without coding knowledge
    • Train GAN or style transfer models
    • Colourize images and videos
    • Create videos from a single image
  • Free. Cost of training time is calculated per minute.

Google DeepMind* Veo2

  • Text prompt* based AI video Generator
  • [Key features]
    • Ability to generate video from text prompts
    • Can use ‘style transfer’ to apply the style of one image or video to a new AI generated video
    • Can be used to create video from still images
    • Can be used to extend the length of video clips
    • Can generate storyboards from text prompts

HunyuanVideo

  • Text prompt* based AI video Generator
  • [Key features]
    • Ability to generate video from text prompts
    • Can use ‘style transfer’ to apply the style of one image or video to a new AI generated video
    • Can be used to create video from still images
    • Can be used to extend the length of video clips
    • Can generate storyboards from text prompts

Hailuoai AI

  • Text prompt* based AI video Generator
  • [Key features]
    • Ability to generate video from text prompts
    • Can use ‘style transfer’ to apply the style of one image or video to a new AI generated video
    • Can be used to create video from still images
    • Can be used to extend the length of video clips
    • Can generate storyboards from text prompts

Kling AI

  • Text prompt* based AI video Generator
  • [Key features]
    • Ability to generate video from text prompts
    • Can use ‘style transfer’ to apply the style of one image or video to a new AI generated video
    • Can be used to create video from still images
    • Can be used to extend the length of video clips
    • Can generate storyboards from text prompts

Comfy UI

  • Node based generative AI platform
  • [Key features]
    • Allows users to create generate AI images and videos using node based programming
    • Free and open source
  • Free

Descript

  • AI text to speech synthesis
  • [Key features]
    • Ability to create an AI natural language emulator based on samples of other voices
    • Text to speech synthesis using customized AI voices
    • Edit audio files of speech using AI
  • Paid with subscription

Case Study

Deep Mind and the Protein Folding Problem

An ongoing scientific project which has engaged researchers for the past 50 years is the ‘protein folding problem’, which has sought to predict the three-dimensional structure of a protein based on its amino acid sequence. Through the work of molecular biologists over the decades, the structures of 100,000 proteins have been discovered, but there are still billions more proteins to be studied. To determine the structure of a single protein, it will take a PhD student or researcher years of painstaking work using current methods, and thus to fully unlock the knowledge contained in the understanding of proteins at the current rate, it may take humans thousands of years. In 2020, the AI company Deep Mind led by Nobel Prize winning scientist Demis Hassabis released AlphaFold2, a deep learning algorithm designed to computationally predict the structure of proteins, and when it was deployed at the Critical Assessment of Structure Prediction event in 2020, a biennale competition in which teams compete to predict the structure of proteins, AlphaFold2 achieved an unprecedented level of accuracy. Within a few months of this event, the number of known protein structures went from 200,000 to 200million, with the problem effectively solved. This example is just the tip of the iceberg, representing the beginning of exponential increases in scientific discovery which will come about as the result of AI.

Protein structure predicted by AlphaFold (Jumper, 2021)

Campus Resources

[Table]

  • Items
  • Location

ChatGPT chatbot

ITSC Virtual Assistant

Glossary

[Glossary]

Artificial General Intelligence (AGI)

  • Artificial general intelligence is the creation of a level of computation intelligence which is on a species level difference to current standards of human intelligence.

Dataset

  • The contents of the data used for training can be text, images, video or audio.

Deep Learning

  • Any approach to AI which involves neural networks.

Foundation Model

  • A deep learning AI model based on massive quantities of unlabeled data.

Generative AI

  • Generative AI is any machine learning system which can generate original text, images, voice, music and video using LLM’s and transformer networks.

Generative Adversarial Network (GAN)

  • A type of AI algorithm used to generate new images and video based on a training data set of prior images and video using deep learning approaches.

Large Language Model

  • A large language model is an AI algorithm used to predict and produce natural language based on enormous amounts of text based training data. Large language models are the basis of AI chatbots like chatPGT.

Natural Language Processing

  • Natural language processing is the ability for an AI model to understand and produce human like speech and conversation.

Neural Network

  • A deep learning approach to AI that is based on the architecture of the neural networks of the human brain using interconnected layers of nodes which resemble the structure of neurons in the brain.

Machine Vision

  • The ability for an AI model to recognize objects in the real world.

Supervised Learning

  • An approach to AI deep learning that uses already labeled data to generate predictions and recognize patterns.

Training

  • The process of using an AI alogorithm to label and classify data to produce a foundation model.

Transformer Network

  • A transformer network is a an artificial intelligence architecture based on deriving meaning from attention and context within large language models.

Unsupervised Learning

  • An approach AI deep learning that can generate predictions and recognize patterns based on unlabeled data.

Back to top