ChatGPT gives accurate advice to breast cancer patients 88% of the time
AN AI chatbot offers correct breast cancer advice 88 per cent of the time, a study shows.
ChatGPT is an online robot that responds to questions like a human and has already been used by more than 100million people since its launch last November.
The bot gave “pretty amazing” answers to questions on breast screening, US researchers found.
Dr Paul Yi, of the University of Maryland, said: “It also has the added benefit of summarising information into an easily digestible form for consumers to easily understand.”
But the bot did give some out-of-date responses, incorrectly advising women to delay getting a mammogram for up to six weeks after getting a Covid vaccine.
Dr Yi said: “Somewhere close to 90 per cent of responses were actually appropriate, which is a good thing.
“But that 10 per cent that were not appropriate, either Chat GPT gave inconsistent responses or the responses were just flat-out wrong.”
Around 55,000 women and 370 men are diagnosed with breast cancer every year in the UK.
Breast screening mammograms are offered to every woman every three years between the ages of 50 and 71.
ChatGPT is a robot launched by OpenAI, a lab set up by tech high-fliers including Elon Musk, that uses one of the most sophisticated language models ever developed.
It has already been adopted by a range of companies to automate emails and even some news sites to write content.
OpenAI upgraded it to be able to browse the internet for select users last month, having previously been reliant on data from before September 2021.
The research, published in Radiology, looked at how effective the bot was at offering advice on breast cancer risk and screening.
Researchers asked it 25 questions, with each repeated three times to see how its responses varied.
In one example, researchers asked what their risk of the disease is.
What are the symptoms of breast cancer?
The first symptom of breast cancer that most women notice is a lump or an area of thickened tissue in their breast.
Most breast lumps are not cancerous, but it’s always best to have them checked by a doctor.
You should see a GP if you notice any of the following:
- a new lump or area of thickened tissue in either breast that was not there before
- a change in the size or shape of one or both breasts
- a discharge of fluid from either of your nipples
- a lump or swelling in either of your armpits
- a change in the look or feel of your skin, such as puckering or dimpling, a rash or redness
- a rash (like eczema), crusting, scaly or itchy skin or redness on or around your nipple
- a change in the appearance of your nipple, such as becoming sunken into your breast
Breast pain is not usually a symptom of breast cancer.
Source: The NHS
The bot said it was unable to tell them their personal risk because it did not have access to their medical records.
But it gave answers that “check out from a medical standpoint”, Dr Yi said, including advising on generic risk factors like a family history of the disease.
In total, responses were correct for 22 out of 25 questions.
Alongside the out-of-date wrong answer, two questions had inconsistent responses that varied significantly each time the same query was posed.
Dr Yi said: “I think these pitfalls indicate that obviously ChatGPT has got a lot of potential but it’s not ready for prime time.
“I think there’s a lot of work that needs to be done that will require collaboration between computer scientists as well as with doctors.”
He said “if we’re going to roll this out and let patients use it, we need to make sure there are these guard rails”.