지난번에도 비슷한 내용으로 게시물을 남긴적이 있는데...
https://www.2cpu.co.kr/vm/13758
수동으로 전이중 설정을 하고 몇달간 잘 돌아가다가
오늘 아침에 다시 같은 현상이 발생했습니다.
수정 전이중 설정을 다시 자동협상으로 변경하니 살아났습니다.
그래서 다시 자동을 수동전이중으로 바꾸니 또 잘 돌아갑니다.
로그파일들을 쭉 보고 있는데 그냥... 갑자기 네트워크가 나간거 밖에는 알수가 없습니다.
도대체 원인이 무엇일까요
지금 생각해보면 서버가 거의 10년 가까이 되었고 (2013년산)
네트워크 포트가 오래되서 죽는게 아닌가..
예상을 해봅니다.
다행히 내부네트워크는 죽지않아 살릴수 있는게
그나마 다행이라 생각됩니다.
흠....
로그를 올려봅니다.
2022-08-29T01:13:19Z dcbd: [info] Ignoring vmnic0 link state change, no port found
2022-08-29T01:13:22Z dcbd: [info] Ignoring vmnic0 link state change, no port found
2022-08-29T01:13:23Z dcbd: [info] Ignoring vmnic0 link state change, no port found
2022-08-29T01:13:23Z dcbd: [info] Ignoring vmnic0 link state change, no port found
2022-08-29T01:13:23Z dcbd: [info] Ignoring vmnic0 link state change, no port found
2022-08-29T01:13:23Z dcbd: [info] Ignoring vmnic0 link state change, no port found
2022-08-29T01:13:35Z dcbd: [info] Ignoring vmnic0 link state change, no port found
2022-08-29T01:13:38Z dcbd: [info] Ignoring vmnic0 link state change, no port found
2022-08-29T01:13:43Z dcbd: [info] Ignoring vmnic0 link state change, no port found
2022-08-29T01:13:43Z dcbd: [info] Ignoring vmnic0 link state change, no port found
2022-08-29T01:13:43Z dcbd: [info] Ignoring vmnic0 link state change, no port found
2022-08-29T01:13:43Z dcbd: [info] Ignoring vmnic0 link state change, no port found
2022-08-29T00:27:10.486Z cpu15:2097902)DVFilter: 5963: Checking disconnected filters for timeouts
2022-08-29T00:35:38.385Z cpu12:2097916)ScsiDeviceIO: 3435: Cmd(0x45a240206980) 0x1a, CmdSN 0xf82c9 from world 0 to dev "naa.600062b2000b9dc02a0562860aa87172" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2022-08-29T00:37:10.486Z cpu15:2097902)DVFilter: 5963: Checking disconnected filters for timeouts
2022-08-29T00:46:52.822Z cpu13:2097916)ScsiDeviceIO: 3435: Cmd(0x45a2402623c0) 0x85, CmdSN 0x14866 from world 2539289 to dev "naa.600062b2000b9dc02a0563901a86de20" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2022-08-29T00:47:10.486Z cpu21:2097902)DVFilter: 5963: Checking disconnected filters for timeouts
2022-08-29T00:53:19.832Z cpu11:2097915)ScsiDeviceIO: 3435: Cmd(0x459a5f5fefc0) 0x1a, CmdSN 0xf832e from world 0 to dev "naa.600062b2000b9dc02a0562860aa87172" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2022-08-29T00:57:10.486Z cpu20:2097902)DVFilter: 5963: Checking disconnected filters for timeouts
2022-08-29T01:05:08.818Z cpu15:2499670)NetSched: 717: 0x2000002: received a force quiesce for port 0x2000017, dropped 18 pkts -- 죽은시점...
2022-08-29T01:05:13.779Z cpu14:2454811)NetSched: 717: 0x2000002: received a force quiesce for port 0x2000013, dropped 427 pkts
2022-08-29T01:05:15.362Z cpu6:2454720)NetSched: 717: 0x2000002: received a force quiesce for port 0x2000012, dropped 35 pkts
2022-08-29T01:07:10.486Z cpu12:2097902)DVFilter: 5963: Checking disconnected filters for timeouts
2022-08-29T01:10:02.991Z cpu17:2454812)NetSched: 717: 0x2000002: received a force quiesce for port 0x2000013, dropped 620 pkts
2022-08-29T01:10:19.086Z cpu6:2454810)NetSched: 717: 0x2000002: received a force quiesce for port 0x2000013, dropped 77 pkts
2022-08-29T01:13:22.051Z cpu16:2097764)<6>igb: vmnic0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None --- 자동협상으로
2022-08-29T01:13:23.486Z cpu3:2097268)NetqueueBal: 5035: vmnic0: device Up notification, reset logical space needed
2022-08-29T01:13:23.486Z cpu3:2097268)NetPort: 1580: disabled port 0x2000002
2022-08-29T01:13:23.486Z cpu14:2539523)NetSched: 654: vmnic0-0-tx: worldID = 2539523 exits
2022-08-29T01:13:23.486Z cpu3:2097268)Uplink: 11689: enabled port 0x2000002 with mac 00:25:90:08:5c:d2
2022-08-29T01:13:23.486Z cpu3:2097268)NetPort: 1580: disabled port 0x2000002
2022-08-29T01:13:23.486Z cpu3:2097268)Uplink: 11689: enabled port 0x2000002 with mac 00:25:90:08:5c:d2
2022-08-29T01:13:23.486Z cpu22:3193581)NetSched: 654: vmnic0-0-tx: worldID = 3193581 exits
2022-08-29T01:13:38.879Z cpu13:2097764)<6>igb: vmnic0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None -- 다시 수동협상으로
그런경우라면 새로 랜카드를 추가하는 방법이 나을수도 있겠습니다.
저도 이런적이 있었는데 원인은 파워서플 이었습니다.
한번 교체해서 모니터링 덤프해보심 파워문제일수도 있지 않을까 싶습니다.
다만 서버내 하드디스크는 전량 SSD로 구성되어 있어서
전기사용량이 그렇게 크지는 않습니다.
파워교체는 케이스의 폐쇄적인 특성상 가능하지 않습니다 ㅠㅠ
신규구매한 DELL서버로 ESXI 6.7 서버를 구성해 운영중인데
이런 문제가 발생한 경우가 전혀 없었습니다.
스위치 문제는 아닌것 같습니다.