[VMWare] vSphere 6.7 업그레이드 후에 vSAN 쓰기 실패되며 vSAN 운영 불가능해지는 문제

포우   
   조회 8051   추천 0    

  센터 및 호스트 업그레이드 후에 "Physical disk health retrieval issues"" 발생했습니다. 기존에 이미 생성된 파일만 수정되는 듯하고, 하기와 같이 디스크 파티션 이레이즈 하고서 새롭게 인식시켜도 이니셜라이즈 안됩니다.

  저는 현재 vSAN의 모든 VM을 비상용 로컬 스토리지로 탈출시켜서 HA 동작 없이 가동 중인 상태입니다. 컨슈머 하드웨어로 운용하는 분은 주의 바랍니다.

vmkernel.log

2018-05-04T01:44:47.807Z cpu0:2098609)FSS: 6522: Conflict between buffered and unbuffered open (file 't10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________'):flags 0x4005, requested flags 0x8001

2018-05-04T01:44:47.807Z cpu0:2098609)Resv: 407: Executed out-of-band reserve on t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________

2018-05-04T01:44:47.809Z cpu0:2098609)FSS: 6522: Conflict between buffered and unbuffered open (file 't10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________'):flags 0x4005, requested flags 0x8001

2018-05-04T01:44:47.809Z cpu0:2098609)Resv: 407: Executed out-of-band release on t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________

2018-05-04T01:44:47.887Z cpu0:2098609)WARNING: Partition: 2368: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: Can't overwrite contents of active f9 partition 2

2018-05-04T01:44:57.896Z cpu0:2098609)WARNING: Partition: 2368: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: Can't overwrite contents of active f9 partition 2

2018-05-04T01:45:07.904Z cpu0:2098609)WARNING: Partition: 2368: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: Can't overwrite contents of active f9 partition 2

2018-05-04T01:45:17.914Z cpu1:2098609)WARNING: Partition: 2368: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: Can't overwrite contents of active f9 partition 2

2018-05-04T01:45:27.921Z cpu1:2098609)WARNING: Partition: 2368: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: Can't overwrite contents of active f9 partition 2

2018-05-04T01:45:37.930Z cpu1:2098609)WARNING: Partition: 2368: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: Can't overwrite contents of active f9 partition 2

2018-05-04T01:45:47.938Z cpu3:2098609)WARNING: Partition: 2368: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: Can't overwrite contents of active f9 partition 2

2018-05-04T01:45:57.949Z cpu3:2098609)WARNING: Partition: 2368: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: Can't overwrite contents of active f9 partition 2

2018-05-04T01:46:07.960Z cpu3:2098609)WARNING: Partition: 2368: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: Can't overwrite contents of active f9 partition 2

2018-05-04T01:46:17.969Z cpu3:2098609)WARNING: Partition: 2368: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: Can't overwrite contents of active f9 partition 2

2018-05-04T01:46:27.978Z cpu0:2098609)WARNING: Partition: 2368: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: Can't overwrite contents of active f9 partition 2

2018-05-04T01:46:37.987Z cpu0:2098609)WARNING: Partition: 2368: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: Can't overwrite contents of active f9 partition 2

2018-05-04T01:46:47.996Z cpu0:2098609)WARNING: Partition: 2368: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: Can't overwrite contents of active f9 partition 2

2018-05-04T01:46:58.004Z cpu0:2098609)WARNING: Partition: 2368: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: Can't overwrite contents of active f9 partition 2

2018-05-04T01:47:08.014Z cpu1:2098609)WARNING: Partition: 2368: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: Can't overwrite contents of active f9 partition 2

2018-05-04T01:47:18.023Z cpu2:2098609)WARNING: Partition: 2368: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: Can't overwrite contents of active f9 partition 2

2018-05-04T01:47:28.027Z cpu1:2098609)FSS: 6522: Conflict between buffered and unbuffered open (file 't10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________'):flags 0x4005, requested flags 0x8001

2018-05-04T01:47:28.027Z cpu1:2098609)Resv: 407: Executed out-of-band reserve on t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________

2018-05-04T01:47:28.036Z cpu1:2098609)FSS: 6522: Conflict between buffered and unbuffered open (file 't10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________'):flags 0x4005, requested flags 0x8001

2018-05-04T01:47:28.037Z cpu1:2098609)Resv: 407: Executed out-of-band release on t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________

2018-05-04T01:47:28.153Z cpu4:2144123)WARNING: Partition: 1922: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: in-use partition 1 modification is not supported

2018-05-04T01:47:33.157Z cpu1:2098609)FSS: 6522: Conflict between buffered and unbuffered open (file 't10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________'):flags 0x4005, requested flags 0x8001

2018-05-04T01:47:33.157Z cpu1:2098609)Resv: 407: Executed out-of-band reserve on t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________

2018-05-04T01:47:33.167Z cpu1:2098609)FSS: 6522: Conflict between buffered and unbuffered open (file 't10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________'):flags 0x4005, requested flags 0x8001

2018-05-04T01:47:33.167Z cpu1:2098609)Resv: 407: Executed out-of-band release on t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________

2018-05-04T01:47:33.265Z cpu0:2144134)WARNING: Partition: 1922: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: in-use partition 1 modification is not supported

2018-05-04T01:47:38.267Z cpu0:2098609)FSS: 6522: Conflict between buffered and unbuffered open (file 't10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________'):flags 0x4005, requested flags 0x8001

2018-05-04T01:47:38.267Z cpu0:2098609)Resv: 407: Executed out-of-band reserve on t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________

2018-05-04T01:47:38.277Z cpu0:2098609)FSS: 6522: Conflict between buffered and unbuffered open (file 't10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________'):flags 0x4005, requested flags 0x8001

2018-05-04T01:47:38.277Z cpu0:2098609)Resv: 407: Executed out-of-band release on t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________

2018-05-04T01:47:38.377Z cpu0:2144145)WARNING: Partition: 1922: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: in-use partition 1 modification is not supported

2018-05-04T01:47:43.195Z cpu1:2097178)ScsiDeviceIO: 3015: Cmd(0x459a40c31c40) 0x1a, CmdSN 0x60f3 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.

2018-05-04T01:47:43.379Z cpu0:2098609)FSS: 6522: Conflict between buffered and unbuffered open (file 't10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________'):flags 0x4005, requested flags 0x8001

2018-05-04T01:47:43.380Z cpu0:2098609)Resv: 407: Executed out-of-band reserve on t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________

2018-05-04T01:47:43.388Z cpu4:2098609)FSS: 6522: Conflict between buffered and unbuffered open (file 't10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________'):flags 0x4005, requested flags 0x8001

2018-05-04T01:47:43.389Z cpu4:2098609)Resv: 407: Executed out-of-band release on t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________

2018-05-04T01:47:43.491Z cpu0:2144163)WARNING: Partition: 1922: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: in-use partition 1 modification is not supported

2018-05-04T01:47:48.493Z cpu4:2098609)FSS: 6522: Conflict between buffered and unbuffered open (file 't10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________'):flags 0x4005, requested flags 0x8001

2018-05-04T01:47:48.494Z cpu4:2098609)Resv: 407: Executed out-of-band reserve on t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________

2018-05-04T01:47:48.503Z cpu1:2098609)FSS: 6522: Conflict between buffered and unbuffered open (file 't10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________'):flags 0x4005, requested flags 0x8001

2018-05-04T01:47:48.503Z cpu1:2098609)Resv: 407: Executed out-of-band release on t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________

2018-05-04T01:47:48.605Z cpu5:2144187)WARNING: Partition: 1922: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: in-use partition 1 modification is not supported

2018-05-04T01:47:53.607Z cpu1:2098609)FSS: 6522: Conflict between buffered and unbuffered open (file 't10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________'):flags 0x4005, requested flags 0x8001

2018-05-04T01:47:53.607Z cpu1:2098609)Resv: 407: Executed out-of-band reserve on t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________

2018-05-04T01:47:53.616Z cpu1:2098609)FSS: 6522: Conflict between buffered and unbuffered open (file 't10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________'):flags 0x4005, requested flags 0x8001

2018-05-04T01:47:53.616Z cpu1:2098609)Resv: 407: Executed out-of-band release on t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________

2018-05-04T01:47:53.716Z cpu3:2144204)WARNING: Partition: 1922: t10.ATA_____SanDisk_SDSSDHII240G____________________143104600201________: in-use partition 1 modification is not supported
하얀고니 2018-05
vSAN 서비스를 재시작해도 동일한 문제가 발생하나요?
     
하림 2018-05
옙. 기존 KB에서 언급되는 서비스를 재시작하거나 호스트를 리부트 해봐도 동일합니다.
          
하얀고니 2018-05
vmkernel로그와 vsan로그 파일 확인이 가능할까요. 원인이 궁금해지네요
               
하림 2018-05
vCenter에서는 적당한 로그 파일을 모르겠네요. 우선 한 호스트에서 새 디스크 그룹 생성 시 과정에서의 로그 캡쳐하여 공유 드립니다. 그 밖에 보고싶은 자료 있다면 말씀주셔요.

디스크 그룹 생성은 51분 27초에 2개의 작업 시작되어 각각 54분  58초와 55분 27초에 끝났습니다. vSAN에서 디스크 관리에서 디스크 추가나 제거가 동작할 때 디스크를 열어서 작업 진행을 못하는한데, 저는 지식이 얕아서 이유까지는 오리무중이네요.
https://www.dropbox.com/sh/cuzzudf1w4bhega/AADV5jNGVXdLGAnetMH3UK5Aa?dl=0
                    
하얀고니 2018-05
쪽지드렸습니다
epowergate 2018-05
이런거 해결하려 하시면 늙어요
그냥 MA 업체 불러서 해결하심이 좋습니다.
     
하림 2018-05
요새는 며칠 밤새면 조금만 자야지 하고서 못일어나는 때가 올것만 같더라고요.
하림 2018-05
결론만 말하면 롤백했습니다. 여러가지 수단을 써보았는데, 6.7에서 가동이 불가능한 상황이네요. NVMe 든 AHCI든 BIOS 레이드 유닛이든 디스크 I/O가 불가능했습니다. 단순한 특정 드라이버 문제가 아닌듯합니다.
하림 2018-05
6.7에서 ESXi 기본 도구를 사용하여 6.5로 롤백하면, 6.7 패키지가 남아서 디펜던시 에러로 인하여 HA 가동 실패합니다. 주의하세요. 이 상태에서는 패키지 설치나 제거 등도 안됩니다.
로그인 하시면 댓글을 남길 수 있습니다


가상화
쓰기
제목Page 56/130
2014-05   5258537   1 정은준1
2015-12   1783370   1  백메가
2017-08   6458   1 arkx
2018-05   9277   1  송주환
2019-01   5419   1 chotws
2019-04   7599   1 나는곰인뎅
2021-01   4181   1 하이바
2021-04   4943   1 쪼커
2021-06   4485   1 야후
2023-11   4365   1 kjh351
2015-03   5733   1 민사장
2018-10   8176   1 Dooood
2018-11   4987   1 통통9
2019-01   5435   1 스모모군
2020-02   5037   1 재니짱
2022-09   9747   1 머스트
2013-11   111213   1 유병국
2014-02   12604   1 니포
2015-02   9374   1 chotws
2015-09   4874   1  깜박깜박가
2016-10   6682   1 행복하세
2017-01   9787   1 서울사람
목록
쓰기