SAP

How to Fix the Hana Log Volume Disk Full Event

by

Learn about the causes and solutions for the Hana log volume disk full event. Find out how to resolve the issue and prevent it from happening in the future.

Symptom

You notice that the log volume is full, and the database does not accept any new requests. Additionally, the trace of the service with the full log volume contains DiskFullEvent messages:

LoggerImpl.cpp : Logger notified of new DiskFull: DiskFullEvent[id= XXX, path= /hana/log/<SID>/mntXXXXX/hdbXXXXX/, state= NEW]

Reason:

Due to an architectural limitation, it was not possible to successfully perform a log backup when no space in the log area was available.

Solution

Apply one of the following SAP HANA Revisions:

  • SAP HANA 2:
    • Revisions >= 060.00 (SPS06)
    • or higher

Workaround:

  1. Make sure that you are logged in as <SID>adm user on the host on which you experience the log volume full situation.
  2. Stop the database:

    /usr/sap/hostctrl/exe/sapcontrol -nr <instance_number> -function StopSystem HDB
  3. Wait that the database on all hosts is stopped completely.
  4. Change directory into folder mnt00001 of the log volume (Default: /hana/log/<SID>/), e.g.:

    cd /hana/log/<SID>/mnt00001
  5. You have to move one of the log volumes temporarily to another volume where enough space is available.
    You should free at least 2 GB of space to ensure that the database has enough space to start. To find out the space consumption of each volume execute:

    du -sh *
  6. Move a volume which consumes at least 2 GB of space (e.g. hdb00003) to a volume with sufficient free space, e.g. to the data volume (Default: /hana/data/<SID>), e.g.:

    mkdir /hana/data/<SID>/logs_temporary

    mv hdb00003 /hana/data/<SID>/logs_temporary
  7. Create a symbolic link to the new folder in the old location:

    ln -s /hana/data/<SID>/logs_temporary/hdb00003 /hana/log/<SID>/mnt00001/hdb00003
  8. Start the database:

    /usr/sap/hostctrl/exe/sapcontrol -nr <instance_number> -function StartSystem HDB
  9. Wait until log backups are performed.
  10. Use the following SQL-Statement on SYSTEMDB and tenant level to clean up the log volume:

    ALTER SYSTEM RECLAIM LOG;
  11. Stop the database again:

    /usr/sap/hostctrl/exe/sapcontrol -nr <instance_number> -function StopSystem HDB
  12. Wait that the database on all hosts is stopped completely.
  13. Remove the symbolic link:

    rm -f /hana/log/<SID>/mnt00001/hdb00003
  14. Move the log volume back to its original location:

    mv /hana/data/<SID>/logs_temporary/hdb00003 /hana/log/<SID>/mnt00001
  15. Remove the temporary log folder:

    rmdir /hana/data/<SID>/logs_temporary
  16. Start the database

    /usr/sap/hostctrl/exe/sapcontrol -nr <instance_number> -function StartSystem HDB

Leave a Comment