Vision Transformer
A Vision Transformer (ViT) applies the Transformer architecture to image recognition by treating image patches as tokens. ViTs have matched or exceeded CNNs on many computer vision benchmarks.
Understanding Vision Transformer
The Vision Transformer (ViT) adapts the transformer architecture from natural language processing to computer vision by treating images as sequences of fixed-size patches, each linearly embedded as a token. This approach demonstrated that pure self-attention models, without any convolutional layers, can achieve state-of-the-art results on image classification benchmarks when trained on sufficient data. ViT divides an image into a grid of patches (typically 16x16 pixels), flattens them into vectors, and processes them through standard transformer encoder layers with multi-head attention. The architecture scales exceptionally well, with larger models and datasets yielding continued performance improvements. Vision transformers have expanded beyond classification to object detection, semantic segmentation, and image generation, challenging the long-standing dominance of CNNs in computer vision tasks.
Category
Computer Vision
Is AI recommending your brand?
Find out if ChatGPT, Perplexity, and Gemini mention you when people search your industry.
Check your brand — $9Related Computer Vision Terms
Bounding Box
A bounding box is a rectangular border drawn around an object in an image to indicate its location and extent. Bounding boxes are the primary output format for object detection models.
Computer Vision
Computer vision is a field of AI that enables machines to interpret and understand visual information from images and videos. Applications include facial recognition, autonomous driving, medical imaging, and augmented reality.
Face Recognition
Face recognition is a computer vision technology that identifies or verifies individuals by analyzing facial features in images or video. It is used in security systems, phone unlocking, and photo organization.
Image Captioning
Image captioning is the AI task of generating natural language descriptions of images. It requires both visual understanding (computer vision) and text generation (NLP) capabilities.
Image Classification
Image classification is the computer vision task of assigning a label to an entire image based on its visual content. Deep learning models like ResNet and Vision Transformers achieve near-human accuracy on this task.
Image Segmentation
Image segmentation is the process of partitioning an image into meaningful regions or classifying each pixel into a category. It is used in medical imaging, autonomous driving, and satellite analysis.
Instance Segmentation
Instance segmentation is a computer vision task that identifies each object in an image and delineates its exact pixel boundary. Unlike semantic segmentation, it distinguishes between individual instances of the same class.
Masked Autoencoder
A masked autoencoder is a self-supervised learning method that masks random patches of an image and trains the model to reconstruct them. It has proven highly effective for pre-training vision models.