9 mind-blowing things you can do with ChatGPT-4 Vision

This allows GPT-4 to handle not only text inputs but images as well, though at the moment it can still only respond in text. It is this functionality that Microsoft said at a recent AI event could eventually allow GPT-4 to process video input into the AI chatbot model. LONDON (AP) — The company behind the ChatGPT chatbot has rolled out its latest artificial intelligence model, GPT-4, in the next step for a technology that’s caught the world’s attention. One of Anthropic’s smaller models, Haiku, has also been turning heads with its performance on the leaderboard. I then uploaded a US dollar to Euro currency conversion chart covering a period of one year. ChatGPT accurately described the chart and even provided some analysis of the value of the US dollar compared to the Euro.

• GPT-4 is “still not fully reliable” because it “hallucinates” facts and makes reasoning errors, it said.
• She referred to the TV show “Black Mirror,” which focuses on the dark side of technology.
• With version 4, you can still use text, but you can also offer an image or even a voice command to make a request from the application.

GPT-4 demonstrates superiority over GPT-3.5, even using human academic achievement standards. OpenAI reported that GPT-3.5 passed the Uniform Bar Exam with a score that would rank in the 10th percentile of test takers compared to actual people aspiring to become lawyers. GPT-4 performed remarkably better, ranking in the 90th percentile compared to humans [3]. Similarly, GPT-3.5 ranked in the 31st percentile compared to high school students competing in the Biology Olympiad, and GPT-4 scored significantly better, scoring in the 99th percentile [3]. Upgrade your life with a daily dose of the biggest tech news, lifestyle hacks and our curated analysis.

In practice, ChatGPT can help you compose pieces of writing, such as personal correspondence, marketing materials, or webpage content. You can ask ChatGPT to help you write a grocery list, create a meal plan, design a workout program, describe art, summarize books, complete math problems, suggest code, or translate between languages, among many other things. “Great care should be taken when using language model outputs, particularly in high-stakes contexts,” the company said, though it added that hallucinations have been sharply reduced.

ChatGPT

Andy’s degree is in Creative Writing and he enjoys writing his own screenplays and submitting them to competitions in an attempt to justify three years of studying. At this time, there are a few ways to access the GPT-4 model, though they’re not for everyone. If you haven’t been using the new Bing with its AI features, make sure to check out our guide to get on the waitlist so you can get early access. It also appears that a variety of entities, from Duolingo to the Government of Iceland have been using GPT-4 API to augment their existing products.

You can also speak to ChatGPT-4, and the AI will generate a voice to speak to you. Multimodality also allows ChatGPT to handle more functions, like captioning or translating videos. When OpenAI released ChatGPT-3.5 to the public in late 2022, it sparked excitement and fear surrounding the breakthrough technology.

In conclusion, ChatGPT-4 Vision is a powerful tool with many applications, from image analysis to educational planning. It’s an exciting development in the field of AI, and I look forward to seeing how it advances. Compare ChatGPT 3.5 versus 4 and learn how the newest version of OpenAI’s generative AI technology is safer, more accurate, and more powerful.

The organization chose not to reveal the specific details of how they trained GPT-4, partially because OpenAI now operates a for-profit arm, and competition in the space is hotter than when GPT-3.5 debuted. Rumors circulated that OpenAI used more than 100 trillion parameters to train GPT-4, but OpenAI CEO Sam Altman strongly denied those rumors. Aside from the new Bing, OpenAI has said that it will make GPT available to ChatGPT Plus users and to developers using the API. While OpenAI hasn’t explicitly confirmed this, it did state that GPT-4 finished in the 90th percentile of the Uniform Bar Exam and 99th in the Biology Olympiad using its multimodal capabilities. Both of these are significant improvements on ChatGPT, which finished in the 10th percentile for the Bar Exam and the 31st percentile in the Biology Olympiad.

ChatGPT-4 is multimodal and can respond to visual and audio input.

GPT-4 is capable of handling over 25,000 words of text, allowing for use cases like long form content creation, extended conversations, and document search and analysis. You can also access GPT-4 for free without any particular web browser requirement by using Bing search while logged into a Microsoft account. It’s that simple to use ChatGPT-4 for free, within the Edge browser, at any time. The Microsoft Edge browser offers perhaps one of the best and easiest ways for an average person to access and use ChatGPT-4 for free, without having to pay for ChatGPT-4 access through OpenAI. Best of all, Edge is available for just about every major platform, including Mac, Windows, Linux, iPhone, iPad, and Android.

In this article, you’ll learn more about the improvements OpenAI made between the GPT-3.5 and GPT-4, as well as examples of how it performs more impressively and information on OpenAI’s pricing model. Andy is Tom’s Guide’s Trainee Writer, which means that he currently writes about pretty much everything we cover. He has previously worked in copywriting and content writing both freelance and for a leading business magazine. His interests include gaming, music and sports- particularly Formula One, football and badminton.

These upgrades are particularly relevant for the new Bing with ChatGPT, which Microsoft confirmed has been secretly using GPT-4. Given that search engines need to be as accurate as possible, and provide results in multiple formats, including text, images, video and more, these upgrades make a massive difference. GPT-3 featured over 175 billion parameters for the AI to consider when responding to a prompt, and still answers in seconds. It is commonly expected that GPT-4 will add to this number, resulting in a more accurate and focused response. In fact, OpenAI has confirmed that GPT-4 can handle input and output of up to 25,000 words of text, over 8x the 3,000 words that ChatGPT could handle with GPT-3.5.

OpenAI says GPT-4’s improved capabilities “lead to new risk surfaces” so it has improved safety by training it to refuse requests for sensitive or “disallowed” information. In an online demo Tuesday, OpenAI President Greg Brockman ran through some scenarios that showed off GPT-4’s capabilities that appeared to show it’s a radical improvement on previous versions. Because of Microsoft’s heavy investment in OpenAI, it’s highly likely that each version of Edge will continue to have GPT present, and even new version of GPT available will be rolled out to Edge users with Copilot as well. This means it’s probable that GPT-4.5, GPT-5, etc will also become available to Edge users for free, whenever those versions of GPT are released to the public.

In China, Baidu Inc. is about to unveil its own bot, Ernie, while Meituan, Alibaba and a host of smaller names are also joining the fray. You can foun additiona information about ai customer service and artificial intelligence and NLP. The argument has been that the bot is only as good as the information it was trained on. It claims ChatGPT-4 is more accurate, creative and collaborative than the previous iteration, ChatGPT-3.5, and “40% more likely” to produce factual responses. It’s been criticized for giving inaccurate answers, showing bias and for bad behavior — circumventing its own baked-in guardrails to spew out answers it’s not supposed to be able to give. OpenAI acknowledged that GPT-4 still has limitations and warned users to be careful. GPT-4 is “still not fully reliable” because it “hallucinates” facts and makes reasoning errors, it said.

Finally, I gave it an image of an unbalanced binary tree and an AVL tree and asked it to create a lesson plan for a high school computer science class based on the image. It developed a comprehensive lesson plan, demonstrating its potential as an educational tool. I uploaded an image of a UK census document from 1851 and asked ChatGPT to transcribe it. Google’s similarly capable Gemini Advanced has been gaining traction as well in the AI assistant space. That may put OpenAI on guard for now, but in the long run, the company is prepping new models.

I uploaded a seemingly blank yellow image with a hidden message in a fun final test. ChatGPT successfully read the hidden message written in a color that’s Chat PG barely noticeable to the naked eye. I also used a chart from a recent video about the Tensor G3 chipset, which shows Geekbench 6 multi-core scores.

Claude’s rise may give OpenAI pause, but as Willison mentioned, the GPT-4 family itself (although updated several times) is over a year old. Chatbot Arena is important to researchers because they often find frustration in trying to measure the performance of AI chatbots, whose wildly varying outputs are difficult to quantify. In fact, we wrote about how notoriously difficult it is to objectively benchmark LLMs in our news piece about the launch of Claude 3. For that story,  Willison emphasized the important role of “vibes,” or subjective feelings, in determining the quality of a LLM. Consider taking an online course to take the next step and learn more about using generative AI like ChatGPT. Prompt Engineering for ChatGPT offered by Vanderbilt University, is a beginner-level course that can teach you to use ChatGPT more effectively.

Like ChatGPT, we’ll be updating and improving GPT-4 at a regular cadence as more people use it. ChatGPT-3.5 sparked much debate and enthusiasm over the many things it could do, such as generate text remarkably similar to human-written work, generate code, and solve math problems. ChatGPT-4’s capabilities are so much greater than ChatGPT-3.5 that a Microsoft research team called it an “early (yet still incomplete) version of an artificial general intelligence (AGI) system” [2]. The team pointed out a few potential applications for GPT-4, including in the fields of mathematics, coding, vision, medicine, law, and psychology.

You can also learn to work with large language patterns and chain of thought prompting. Not only was ChatGPT previously limited to information available online before 2021, but it also had a limited short-term memory of about 8,000 words. In conversation, the program might remember about 8,000 words until it starts forgetting what you discussed previously. The latest iteration of the model has also been rumored to have improved conversational abilities and sound more human. Some have even mooted that it will be the first AI to pass the Turing test after a cryptic tweet by OpenAI CEO and Co-Founder Sam Altman.

This feature can be handy for converting graphical data into a more manageable format. This upgraded feature is good news considering the controversies OpenAI is involved with surrounding misinformation, inaccuracy, and bias in its answers. One example of such controversy is a groundbreaking defamation lawsuit where a man claimed ChatGPT falsely reported that he embezzled money.

For a full demonstration of how to use ChatGPT-4 Vision to accomplish these tasks, be sure to watch the video embedded above. Learners are advised to conduct additional research to ensure that courses and other credentials pursued meet their personal, professional, and financial goals. Coursera’s editorial team is comprised of highly experienced professional editors, writers, and fact…

Microsoft also needs this multimodal functionality to keep pace with the competition. Both Meta and Google’s AI systems have this feature already (although not available to the general public). While Microsoft Corp. has pledged to https://chat.openai.com/ pour $10 billion into OpenAI, other tech firms are hustling for a piece of the action. Alphabet Inc.’s Google has already unleashed its own AI service, called Bard, to testers, while a slew of startups are chasing the AI train.

ChatGPT can write silly poems and songs or quickly explain just about anything found on the internet. It also gained notoriety for results that could be way off, such as confidently providing a detailed but false account of the Super Bowl game days before it took place, or even being disparaging to users. “With GPT-4, we are one step closer to life imitating art,” said Mirella Lapata, professor of natural language processing at the University of Edinburgh. She referred to the TV show “Black Mirror,” which focuses on the dark side of technology. These new AI breakthroughs have the potential to transform the internet search business long dominated by Google, which is trying to catch up with its own AI chatbot, and numerous professions.

Training

For example, you could ask ChatGPT-4 to analyze a document for you, and it can now process about 25,000 words at a time. Another version of the technology called ChatGPT-4 Turbo can process up to 128,000 words. With this feature, you could include a website link in your prompt and ask ChatGPT to consider that source when giving its answer. One of ChatGPT-4’s most dazzling new features is the ability to handle not only words, but pictures too, in what is being called “multimodal” technology.

We used GPT-4 to help create training data for model fine-tuning and iterate on classifiers across training, evaluations, and monitoring. Once GPT-4 begins being tested by developers in the real world, we’ll likely see the latest version of the language model pushed to the limit and used for even more creative tasks. In addition to GPT-4, which was trained on Microsoft Azure supercomputers, Microsoft has also been working on the Visual ChatGPT tool which allows users to upload, edit and generate images in ChatGPT. The other major difference is that GPT-4 brings multimodal functionality to the GPT model.

OpenAI says GPT-4 “exhibits human-level performance.” It’s much more reliable, creative and can handle “more nuanced instructions” than its predecessor system, GPT-3.5, which ChatGPT was built on, OpenAI said in its announcement. Remember that OpenAI ChatGPT uses GPT 3 at the free level, and using GPT-4 with OpenAI requires a monthly subscription. The Edge browser and Copilot feature offers GPT-4 entirely for free, since it’s built directly into the browser. The Copilot-based GPT-4 access and capabilities works the same on Microsoft Edge for Mac, Windows, iPhone, iPad, Android, Linux etc, though the interface is a little different on iPhone and iPad. The potential is near limitless, but to understand just how powerful this tech is you really need to use it yourself, and to experiment with it. You can confirm this by directly asking ChatGPT/Copilot a question about it’s version, by typing “what version of GPT are you using?

• These new AI breakthroughs have the potential to transform the internet search business long dominated by Google, which is trying to catch up with its own AI chatbot, and numerous professions.
• This neural network uses machine learning to interpret data and generate responses and it is most prominently the language model that is behind the popular chatbot ChatGPT.
• GPT-3 and GPT-4 are large language models created by OpenAI, the organization that also runs ChatGPT, and other artificial intelligence models like DALL-E.
• Through thousands of these subjective comparisons, Chatbot Arena calculates the “best” models in aggregate and populates the leaderboard, updating it over time.
• Currently, the free preview of ChatGPT that most people use runs on OpenAI’s GPT-3.5 model.
• GPT-4-assisted safety researchGPT-4’s advanced reasoning and instruction-following capabilities expedited our safety work.

It is expected to release a major new successor to GPT-4 Turbo (whether named GPT-4.5 or GPT-5) sometime this year, possibly in the summer. It’s clear that the LLM space will be full of competition for the time being, which may make for more interesting shakeups on the Chatbot Arena leaderboard in the months and years to come. The “vibes” sentiment is common in the AI space, where numerical benchmarks that measure knowledge or test-taking ability are frequently cherry-picked by vendors to make their results look more favorable. “Just had a long coding session with Claude 3 opus and man does it absolutely crush gpt-4. I don’t think standard benchmarks do this model justice,” tweeted AI software developer Anton Bacaj on March 19. “For the first time, the best available models—Opus for advanced tasks, Haiku for cost and efficiency—are from a vendor that isn’t OpenAI,” independent AI researcher Simon Willison told Ars Technica. “That’s reassuring—we all benefit from a diversity of top vendors in this space. But GPT-4 is over a year old at this point, and it took that year for anyone else to catch up.”

GPT stands for generative pre-trained transformers and is the brains and power behind ChatGPT’s capabilities. GPT-3 and GPT-4 are large language models created by OpenAI, the organization that also runs ChatGPT, and other artificial intelligence models like DALL-E. ChatGPT is an application created and managed by OpenAI that allows users to interface with GPT, a large language model that powers generative artificial intelligence. Users can submit requests to the app, and the AI model will consult its training data set to return a unique response to the prompt. Training with human feedbackWe incorporated more human feedback, including feedback submitted by ChatGPT users, to improve GPT-4’s behavior.

A user will have the ability to submit a picture alongside text — both of which ChatGPT-4 will be able to process and discuss. It’s part of a new generation of machine-learning systems that can converse, generate readable text on demand and chat gpt four produce novel images and video based on what they’ve learned from a vast database of digital books and online text. GPT-4-assisted safety researchGPT-4’s advanced reasoning and instruction-following capabilities expedited our safety work.

It may also be what is powering Microsoft 365 Copilot, though Microsoft has yet to confirm this. In it, he took a picture of handwritten code in a notebook, uploaded it to GPT-4 and ChatGPT was then able to create a simple website from the contents of the image. In this portion of the demo, Brockman uploaded an image to Discord and the GPT-4 bot was able to provide an accurate description of it. Currently, if you go to the Bing webpage and hit the “chat” button at the top, you’ll likely be redirected to a page asking you to sign up to a waitlist, with access being rolled out to users gradually. GPT-4 is a “large multimodal model,” which means it can be fed both text and images that it uses to come up with answers.

OpenAI released the next version, ChatGPT-4, in March of 2023 with demonstrated improvements in accuracy, security, memory, context windows, and increased functionality to respond to images and voice prompts. It’s been a mere four months since artificial intelligence company OpenAI unleashed ChatGPT and — not to overstate its importance — changed the world forever. In just 15 short weeks, it has sparked doomsday predictions in global job markets, disrupted education systems and drawn millions of users, from big banks to app developers. This neural network uses machine learning to interpret data and generate responses and it is most prominently the language model that is behind the popular chatbot ChatGPT.

ChatGPT provided a detailed image description, including the smallest elements, demonstrating its ability to interpret and describe complex visuals. One of the impressive features of ChatGPT-4 Vision is its ability to read handwritten notes and diagrams. Despite the poor handwriting and drawing, ChatGPT managed to interpret it accurately and even converted it into Python code.

Currently, the free preview of ChatGPT that most people use runs on OpenAI’s GPT-3.5 model. This model saw the chatbot become uber popular, and even though there were some notable flaws, any successor was going to have a lot to live up to. It’s less likely to answer questions on, for example, how to build a bomb or buy cheap cigarettes. GPT-4 is also “steerable,” which means that instead of getting an answer in ChatGPT’s “classic” fixed tone and verbosity, users can customize it by asking for responses in the style of a Shakespearean pirate, for instance. Generative AI technology like GPT-4 could be the future of the internet, at least according to Microsoft, which has invested at least $1 billion in OpenAI and made a splash by integrating AI chatbot tech into its Bing browser. And together it’s this amplifying tool that lets you just reach new heights,” Brockman said.

GPT-4 is the most recent version of this model and is an upgrade on the GPT-3.5 model that powers the free version of ChatGPT. “We should remember that language models such as GPT-4 do not think in a human-like way, and we should not be misled by their fluency with language,” said Nello Cristianini, professor of artificial intelligence at the University of Bath. ChatGPT-4 also has a longer context window, or the amount of text it can process simultaneously.

However, it’s important to note that ChatGPT should not be used for financial or medical advice. We profiled how the site works in December, but in brief, Chatbot Arena presents a user visiting the website with a chat input box and two windows showing output from two unlabeled LLMs. The user’s task it to rate which output is better based on any criteria the user deems most fit. Through thousands of these subjective comparisons, Chatbot Arena calculates the “best” models in aggregate and populates the leaderboard, updating it over time. According to OpenAI, ChatGPT-4 is “82 percent less likely to respond to requests for disallowed content and 40 percent more likely to produce factual responses than GPT-3.5 on our internal evaluations” [3]. The organization reports that it achieved this new security and accuracy using user feedback, consultations with security experts, and real-world applications.

“The king is dead,” tweeted software developer Nick Dobos in a post comparing GPT-4 Turbo and Claude 3 Opus that has been making the rounds on social media. In the previous version, you needed to write a prompt using text to generate an output from ChatGPT. With version 4, you can still use text, but you can also offer an image or even a voice command to make a request from the application. OpenAI’s example of this new feature is that you could put in a picture of the inside of your refrigerator, and ChatGPT-4 could suggest recipes you could make with the ingredients in the image.