Incident Report

Summary

On Friday, 2 May 2025, Gmail, Google Chat, and Google Drive experienced an issue where some customers using Android devices and web browsers were unable to properly display Portable Document Format (PDF) files globally. Generative AI features associated with the impacted PDF files were also affected, for a duration of 5 days, 12 hours, and 4 minutes.

To our customers whose workflows were impacted during this disruption, we sincerely apologize. This is not the level of quality and reliability we strive to offer you, and we are taking immediate steps to improve the platform's performance and availability.

Root Cause

The root cause of the issue was an update to an underlying software library that affected how PDF files were interpreted during the rendering process.

A recent software update to an underlying, third-party library inadvertently introduced a change in how PDF files were interpreted during the rendering process. This change led to some Gmail, Google Chat, and Google Drive users on Android devices and web browsers seeing incomplete or inaccurate content within certain PDFs. This issue also impacted the generative AI features for the affected PDF files. iOS users were unaffected.

Remediation and Prevention

Google engineers were alerted to the outage on Wednesday, 7 May at 05:52 US/Pacific via a customer support case and immediately started an investigation. Once the root cause was identified, the issue was mitigated by reverting to the previous software version, which restored the correct rendering of PDFs. The incident was fully mitigated on Wednesday, 7 May 16:21 US/Pacific.

Google is committed to preventing a repeat of this issue in the future and is completing the following actions:

• Add comprehensive test coverage for all new features and critical existing functionality.
• Improve release-time validation of code changes.

Detailed Description of Impact

• Google Drive: PDF files in drive were not displayed correctly due to rendering issues. Furthermore, Gemini was also affected by the same rendering issue as the Drive viewer for some PDFs.

• Gmail: PDF files in email were not displayed correctly due to rendering issues.

• Google Chat: PDF files in chat were not displayed correctly due to rendering issues.

Mini Incident Report

We apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced an impact outside of what is listed below, please reach out to Google Workspace Support using the help article https://support.google.com/a/answer/1047213.

(All Times US/Pacific)

Incident Start: 30 Apr 2025, 09:00

Incident End: 7 May 2025, 15:12

Duration: 7 days, 6 hours, 12 minutes

Affected Services and Features:

Google Drive, Gmail, Google Chat

Regions/Zones: Global

Description:

Some PDF files in Google Drive, Gmail and Google Chat were not displayed properly for customers using Android devices and web browsers. The issue also affected the generative AI features for the affected PDF files. This occurred for a duration of 7 days, 6 hours, 12 minutes. The root cause of the incident was a code change in an internal service. This update triggered the incorrect rendering of information in PDFs, impacting users of services dependent on this rendering service. The impact was mitigated by rolling back the internal services to a previous version.

Google will complete a full Incident Report in the following days that will provide a full root cause analysis.

Customer Impact:

Some PDF files in Google Drive, Gmail, and Google Chat were not displayed properly for customers using android devices and web browsers. The issue also affected the generative AI features for the affected PDF files.

The issue did not affect customers using iOS devices.

Summary:

Google Drive, Gmail, and Google Chat customers using Android devices, and Web browser are experiencing rendering issues with some PDF files

Description:

The recent issue with PDF rendering affecting Gmail, Google Chat, Google Drive (Web and Android) have been fully resolved.

Gemini responses for some pdfs which were uploaded to Google Drive between April 23rd and May 7th would have degraded responses in the Gemini side panel on Gmail, Google Chat and Google Drive web apps.

Our engineering team continues to work on mitigating the Gen AI functionality and is expected to complete by Thursday, 2025-05-08 18:00 US/Pacific.

We will provide an update by Thursday, 2025-05-08 19:00 US/Pacific with current details.

Symptoms:

Some PDF files in Google Drive, Gmail, and Google Chat are not being displayed properly for customers using Android devices and Web browsers.

The issue may also affect the Gen AI features for the affected PDF Files.

The issue does not affect customers using iOS devices.

Workarounds:

Customers can download the affected PDF files and view them without any issues. If Gemini response is degraded users can use a copy of original pdf and that would work.

Summary:

Google Drive, Gmail, and Google Chat customers using Android devices, and Web browser are experiencing rendering issues with some PDF files

Description:

We believe the issue with Gmail Web, Google Chat Web, Google Drive Web is fully resolved.

Our engineers are working on mitigating the issue for Gmail, Google Chat and Google Drive Android customers.

Full resolution is expected to complete complete by Wednesday, 2025-05-07 18:30 US/Pacific.

We will provide an update by Wednesday, 2025-05-07 19:00 US/Pacific with current details.

Symptoms:

Some PDF files in Google Drive, Gmail, and Google Chat are not being displayed properly for customers using Android devices and Web browsers.

The issue may also affect the Gen AI features for the affected PDF Files.

The issue does not affect customers using iOS devices.

Workarounds:

Customers can download the affected PDF files and view them without any issues.

Summary:

Google Drive, Gmail, and Google Chat customers using Android devices, and Web browser are experiencing rendering issues with some PDF files

Description:

We believe the issue with Gmail Web , Google Chat Web is fully resolved.

Our engineers are working on mitigating the issue for Gmail and Google Chat Android customers, and Google Drive Android and Web customers.

Full resolution is expected to complete complete by Wednesday, 2025-05-07 15:30 US/Pacific.

We will provide an update by Wednesday, 2025-05-07 15:45 US/Pacific with current details.

Symptoms:

Some PDF files in Google Drive, Gmail, and Google Chat are not being displayed properly for customers using Android devices and Web browsers.

The issue may also affect the Gen AI features for the affected PDF Files.

The issue does not affect customers using iOS devices.

Workarounds:

Customers can download the affected PDF files and view them without any issues.

Summary:

Google Drive, Gmail, and Google Chat customers using Android devices, and Web browser are experiencing rendering issues with some PDF files

Description:

We believe the issue with Gmail , Google Chat Web is fully resolved.

Our engineers are working on mitigating the issue for Gmail and Google Chat Android customers, and Google Drive Android and Web customers.

Full resolution is expected to complete by Wednesday, 2025-05-07 12:30 US/Pacific.

We will provide an update by Wednesday, 2025-05-07 12:45 US/Pacific with current details.

Symptoms:

Some PDF files in Google Drive, Gmail, and Google Chat are not being displayed properly for customers using Android devices and Web browsers.

The issue may also affect the Gen AI features for the affected PDF Files.

The issue does not affect customers using iOS devices.

Workarounds:

Customers can download the affected PDF files and view them without any issues.

Summary:

Google Drive, Gmail, and Google Chat customers are experiencing rendering issues with some PDF files

Description:

We are experiencing an issue with Google Drive, Gmail, and Google Chat beginning at Wednesday, 2025-05-07 07:47 US/Pacific.

Mitigation work is currently underway by our engineering team.

The mitigation is expected to complete by Wednesday, 2025-05-07 11:00 US/Pacific.

We will provide more information by Wednesday, 2025-05-07 11:15 US/Pacific.

Customer Symptoms:

Some PDF files in Google Drive , Gmail, and Google Chat are not being displayed properly

Workaround:

Customers can download the affected PDF files and view them without any issues.

Summary:

Google Drive, Gmail, and Google Chat customers are experiencing rendering issues with some PDF files

Description:

We are experiencing an issue with Google Drive, Gmail, and Google Chat beginning at Wednesday, 2025-05-07 07:47 US/Pacific.

Mitigation work is currently underway by our engineering team.

The mitigation is expected to complete by Wednesday, 2025-05-07 11:00 US/Pacific.

We will provide more information by Wednesday, 2025-05-07 11:15 US/Pacific.

Customer Symptoms:

Some PDF files in Google Drive , Gmail, and Google Chat are not being displayed properly

Workaround:

None at this time.

Summary:

Google Drive and Gmail customers are experiencing rendering issues with some PDF files

Description:

We are experiencing an issue with Google Drive, Gmail beginning at Wednesday, 2025-05-07 07:47 US/Pacific.

Our engineering team continues to investigate the issue.

We will provide an update by Wednesday, 2025-05-07 09:15 US/Pacific with current details.

We apologize to all who are affected by the disruption.

Customer Symptoms:

Some PDF files in Google Drive and Gmail are not being displayed properly

Workaround:

None at this time.