Practical AI Image Analysis in the Workplace

AI now possesses vision.

With the advancement of multimodal technology, which allows AI to process different types of data—including images, videos, and audio—image analysis has become significantly easier and more accessible.

Multimodal refers to technology that processes multiple types of data simultaneously.

In 2023, OpenAI launched GPT Vision, which specializes in image analysis, demonstrating that AI can perform detailed image analysis. This capability has since been integrated into the GPT-4o model.

AI processes and interprets images in three main stages.

The AI breaks the input image into smaller segments and analyzes each part to identify individual elements.

The image is analyzed to find specific patterns and key elements.

Key patterns and characteristics within the image are identified. These distinct elements, known as landmarks, include facial features like eyes, nose, mouth, and ears.

Finally, AI combines the extracted features to understand what the entire image represents.

For example, if an image contains elements like trees, a blue sky, and a person, AI may interpret it as "a person walking in the park."

AI-powered image analysis has a wide range of real-world applications. Here are some common use cases.

AI can be used to extract text from images, for example, extracting phone numbers from a business card image or amounts from a receipt image.

This process of extracting text from images is known as OCR (Optical Character Recognition).

When classifying thousands or even tens of thousands of images, using AI can make the process much faster and more accurate.

AI can analyze images of graphs, charts, and tables to extract data or visualize data by analyzing images.

For instance, it can analyze stock chart images to extract stock prices or analyze map images to visualize population density.

When crafting prompts for image analysis, employing the following methods can yield more accurate results.

Providing background information or related context of the image can lead to better accurate results.

Prompt Example

• This image is a photograph taken in nature. Identify 3 main objects.

• This photo is a business card. Extract the name, job title, and contact information.

• The following graph represents book sales revenue for the second half of 2023. Extract the sales amount and book categories from the graph and organize them in a table.

Instruct the AI to analyze specific parts, text, or objects in the image.

Prompt Example

• Extract the text located in the top right corner of this image.

• Describe the individual in the center of this photo.

• Extract the sales figure for July 2023 in this graph.

It's beneficial to explicitly specify the output format in the prompt, such as CSV (comma-separated values used in spreadsheets), Table, List, Sentence, etc.

Prompt Example

• Organize the extracted values from the graph in CSV format for use in Excel.

• Organize the extracted name, job title, and contact information from the business card into a list.

By applying the methods above, you can draft an image analysis prompt as follows:

Copy
The provided image is a business card.

Please extract the name, job title, contact information, and email from the card.

Organize the extracted information in CSV format.

• Image Context: Business card

• Extraction Details: Name, job title, contact information, email

• Answer Output Format: CSV format

Send a prompt example and compare the AI's responses.