Linux硬重启后磁盘损坏导致MySQL无法启动问题修复
故障现象
服务重启后宝塔面板没法把mysql拉起来了,尝试命令行启动mysql报错
- [root@instance-20221108-1616 ~]# service mysqld start
- Starting MySQL. ERROR! The server quit without updating PID file (/www/server/data/instance-20221108-1616.pid).
故障处理
查看MySQL报错日志，报错找不到mysql-bin.index文件
- 2022-11-08T12:04:40.000499Z 0 [Warning] option 'max_allowed_packet': unsigned value 107374182400 adjusted to 1073741824
- 2022-11-08T12:04:40.000595Z 0 [Note] --secure-file-priv is set to NULL. Operations related to importing and exporting data are disabled
- 2022-11-08T12:04:40.000612Z 0 [Note] /www/server/mysql/bin/mysqld (mysqld 5.7.34-log) starting as process 68663 ...
- mysqld: File './mysql-bin.index' not found (Errcode: 5 - Input/output error)
- 2022-11-08T12:04:40.014762Z 0 [ERROR] Aborting
尝试创建文件也失败了
- [root@instance-20221108-1616 data]# touch mysql-bin.index
- touch: cannot touch 'mysql-bin.index': Input/output error
尝试给文件夹赋权，失败
- [root@instance-20221108-1616 data]# chmod 777 -R /www
- chmod: cannot access '/www/server/data/ibtmp1': Input/output error
- chmod: cannot access '/www/server/data/mysql-bin.index': Input/output error
猜测磁盘损坏
卸载磁盘
- [root@instance-20221108-1616 data]# umount /dev/sdb
- umount: /www: target is busy.
- [root@instance-20221108-1616 data]# umount /www
- umount: /www: target is busy.
- [root@instance-20221108-1616 data]# umount /dev/sdb -l
尝试恢复报错 cannot open /dev/sdb: Device or resource busy
reboot后重新执行（extX 的分区格式使用fsck修复，XFS分区格式使用 mkfs命令修复 mkfs.xfs -f /dev/sdb1）
- [root@instance-20221108-1616 ~]# fsck -t ext2 -y /dev/sdb
- fsck from util-linux 2.32.1
- e2fsck 1.45.6 (20-Mar-2020)
- /dev/sdb contains a file system with errors, check forced.
- Pass 1: Checking inodes, blocks, and sizes
- Deleted inode 4727069 has zero dtime. Fix? yes
- Deleted inode 4727073 has zero dtime. Fix? yes
- Deleted inode 4727074 has zero dtime. Fix? yes
- Deleted inode 7184391 has zero dtime. Fix? yes
- Deleted inode 7184392 has zero dtime. Fix? yes
- Deleted inode 7184393 has zero dtime. Fix? yes
- Deleted inode 7184394 has zero dtime. Fix? yes
- Deleted inode 7184401 has zero dtime. Fix? yes
- Deleted inode 7184402 has zero dtime. Fix? yes
- Deleted inode 7184403 has zero dtime. Fix? yes
- Deleted inode 7184404 has zero dtime. Fix? yes
- Deleted inode 7184405 has zero dtime. Fix? yes
- Deleted inode 7184406 has zero dtime. Fix? yes
- Extended attribute block 18940418 has reference count 1025, should be 1024. Fix? yes
- Extended attribute block 20152834 has reference count 1018, should be 1014. Fix? yes
- Extended attribute block 20254344 has reference count 882, should be 876. Fix? yes
- Pass 2: Checking directory structure
- Entry 'ibtmp1' in /server/data (4046849) has deleted/unused inode 4046850. Clear? yes
- Entry 'mysql-bin.index' in /server/data (4046849) has deleted/unused inode 4046878. Clear? yes
- Pass 3: Checking directory connectivity
- Pass 3A: Optimizing directories
- Pass 4: Checking reference counts
- Unattached inode 4046879
- Connect to /lost+found? yes
- Inode 4046879 ref count is 2, should be 1. Fix? yes
- Unattached inode 4046880
- Connect to /lost+found? yes
- Inode 4046880 ref count is 2, should be 1. Fix? yes
- Pass 5: Checking group summary information
- Block bitmap differences: -(16811631--16811636) -(16811638--16812650) -(16812688--16812690) -(16812744--16812754) -(16812880--16813881) -(16813883--16814923) +16814925 -(16814950--16816127) -(18900624--18900630) -(18900728--18900734) -(18900965--18901086) -(18901088--18901094) -(18901159--18907135) -18907650 -(18907656--18913279) -18916738 -18916749 -(18916757--18916795) -(18916818--18916887) -(18916889--18917068) -(18917075--18917144) -(18917146--18917178) -(18917180--18917325) -(18917327--18917431) -(18917433--18919956) -(18919958--18919965) -18920002 -(18920032--18920033) -(18920078--18920379) +18921709 -18939400 -(28835492--28835839) -(28836354--28838548) -(28854739--28854746) -(28949148--28949152) -28976918 -28976920 -28976956 -(29000195--29000391) -(29000395--29000398) -(29000403--29000466) -(29002907--29003035) -(29003043--29003127) -(29003640--29003775) -(29003795--29004007) -(29004012--29004227) -(29004280--29004323)
- Fix? yes
- Free blocks count wrong for group #513 (1132, counted=5385).
- Fix? yes
- Free blocks count wrong for group #576 (0, counted=6120).
- Fix? yes
- Free blocks count wrong for group #577 (3258, counted=12365).
- Fix? yes
- Free blocks count wrong for group #879 (24717, counted=25065).
- Fix? yes
- Free blocks count wrong for group #880 (25304, counted=27507).
- Fix? yes
- Free blocks count wrong for group #883 (29383, counted=29388).
- Fix? yes
- Free blocks count wrong for group #884 (28891, counted=28894).
- Fix? yes
- Free blocks count wrong for group #885 (6450, counted=7538).
- Fix? yes
- Free blocks count wrong (34182084, counted=34205211).
- Fix? yes
- Inode bitmap differences: -4046850 -4046878 +(4046879--4046880) -4727069 -(4727073--4727074) -(7184391--7184394) -(7184401--7184406)
- Fix? yes
- Free inodes count wrong for group #577 (7853, counted=7856).
- Fix? yes
- Directories count wrong for group #577 (7, counted=6).
- Fix? yes
- Free inodes count wrong for group #877 (8160, counted=8170).
- Fix? yes
- Free inodes count wrong (9705033, counted=9705046).
- Fix? yes
- /dev/sdb: ***** FILE SYSTEM WAS MODIFIED *****
- /dev/sdb: 125354/9830400 files (0.9% non-contiguous), 5116389/39321600 blocks
修复完成，mysqld启动成功
- [root@instance-20221108-1616 ~]# service mysqld start
- Starting MySQL. SUCCESS!
- [root@instance-20221108-1616 ~]# pidof mysqld
- 18335
