ÆÄÀ̽ã Áú¹®

   Á¶È¸ 3606   Ãßõ 0    

안녕하세요

파이썬으로 제가 사용하는 계산을 자동으로 처리하려고 하는데,

지금까지는 제가 이렇게 사용했습니다. (제대로 짠건지는 모르겠지만 일단 작동은 잘되네요).

계산 한번에 10분이 걸린다고 하면 (코어 1개만 사용합니다), 순차적으로 10분씩 걸려서 끝날 때 까지 너무 오래걸립니다.

cpu 코어가 32개라서 동시에 계산을 하고싶은데, 가능한가요??

조언부탁드립니다.

감사합니다.



ªÀº±Û Àϼö·Ï ½ÅÁßÇÏ°Ô.
chis 2020-06
½Ã°£ÀÌ Á» µÈÁö¶ó Àúµµ ±â¾ïÀÌ °¡¹°°¡¹°Çѵ¥¿ä.
ffmpegÀ¸·Î ´Ù¼öÀÇ À½¼ºÆÄÀÏÀ» Æ®·£½ºÄÚµù ÇÏ·Á°í Çߴµ¥ ÄÚ¾î1°³·Î¸¸ ÀÛ¾÷À» ÇÏ´Ï±î ¿À·¡°É·Á¼­
¸ÖƼÄÚ¾î·Î Çغ¼¶ó°í »ðÁúÀ» Á» Çß¾ú´Âµ¥ °á·ÐÀº ½ÇÆÐÇß½À´Ï´Ù..
¿ÜºÎ ÇÁ·Î±×·¥À» ºÒ·¯¿À´Â°Ç ÀÌ»óÇÏ°Ô ¾ÈµÇ´õ¶ó°í¿ä..
¹°·Ð Á¦°¡ ½Ç·ÂÀÌ ¸¹ÀÌ ¹ÌõÇÑ°í·Î °¡´ÉÇÑ°ÍÀ» ¸øÇÑ°ÍÀÏ °¡´É¼ºÀÌ Å©±äÇÏÁö¸¸ Âü°íÇϼ¼¿ä..
Æз¯·¼ ÇÁ·Î±×·¡¹ÖÀ» ÇϼžßÁÒ.
ÀϹÝÀûÀÎ ÇÁ·Î±×·¥µéÀº CPU ÇÑ°³¸¸ ¾¹´Ï´Ù.

https://blog.naver.com/PostView.nhn?blogId=woosoung1993&logNo=221639525899
¹«¾Æ 2020-06
ÇϳªÀÇ ½ºÅ©¸³Æ®·Î for ¹® 10°³ÀÇ ÀÛ¾÷ ÇÒ °ÍÀ»
for ¹® ¾øÀÌ Çϳª¸¸ ÇÒ ¼ö ÀÖ´Â ½ºÅ©¸³Æ®¸¦ 10°³ µû·Î (µ¿½Ã¿¡ ¹é±×¶ó¿îµå·Î) µ¹¸®´Â °ÍÀÌ ÁÁÁö¾ÊÀ»±î¿ä??
Wheein 2020-06
½ÇÁ¦ ÀÏÀº ´Ù¸¥ ÇÁ·Î¼¼½º°¡ ÇÏ°í, ÆÄÀ̽ãÀº stdin, stdout ¸¸ °ü¸®ÇÏ´Â °ÍÀÌ´Ï thread ¸¸ ³ª´©¾îµµ µÉ °Í °°½À´Ï´Ù.

from concurrent.futures import ThreadPoolExecutor

def _do_work(i):
    command = "..."
    p = Popen(command, ...)
    p.communicate(...)

with ThreadPoolExecutor(32) as pool:
    pool.map(_do_work, [110, 120, ...])

´ë°­ ¿ä·± ´À³¦À¸·Î¿ä
Àß ¸¸µå½Å °Í °°Àºµ¥¿ä. ÇÁ·Î¼¼½º´Â 10°³°¡ »ý¼ºµÇ¾î¼­ µ¹¾Æ°¥ÅÙµ¥... ¾Æ¸¶ I/O¶§¹®¿¡ blockµÇ¾î¼­ ±× °á°ú¸¦ pythonÀÌ °¡Á®°¡ ÁÙ ¶§±îÁö ±â´Ù¸®±â ¶§¹®¿¡ ¼øÂ÷ÀûÀ¸·Î µ¹¾Æ°¡´Â °Íó·³ º¸ÀÌ´Â °Í °°½À´Ï´Ù. I/O¸¦ ÆÄÀÏ¿¡¼­ ¹Þ°í ÆÄÀÏ·Î ÀúÀåµÇµµ·Ï ÇÑ´Ù¸é, °¢°¢ÀÇ ÇÁ·Î¼¼½º´Â µû·Î µ¹¾Æ°¥ °Ì´Ï´Ù. ´Ù µ¹°í ÆÄÀÏ¿¡¼­ °á°ú¸¦ È®ÀÎÇÏ¸é µÇ°ÚÁÒ.
ÇÏÁö¸¸, processµéÀÌ °°Àº ¸®¼Ò½º¸¦ °øÀ¯Çؼ­ »ç¿ëÇØ¾ß ÇÑ´Ù¸é, °á°ú´Â µ¿ÀÏÇÒ °Å¶ó°í »ý°¢µË´Ï´Ù.

Àý´ë ±×·² ÀÏÀÌ ¾È »ý±â°Ô ¿¹¸¦ µé¾î ¾Æ·¡Ã³·³ ¹«ÇÑ loopÀ» Çϳª ¸¸µé°í...

root@ubuntu-20:~/python_subprocess# cat loop.c
#include <unistd.h>
#include <stdio.h>

int main(void)
{
  while (1) {
    usleep(1);
  }
}

ÄÄÆÄÀÏ ÇÑ ´ÙÀ½, ÆÄÀ̽㿡¼­ µ¹·Áº¸¸é...
root@ubuntu-20:~/python_subprocess# python3
Python 3.8.2 (default, Apr 27 2020, 15:53:34)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> proc_list = []
>>> for x in range(10):
...    proc_list.append(subprocess.Popen(["./loop"]))
...
>>> proc_list
[<subprocess.Popen object at 0x7f2bdbc332b0>, <subprocess.Popen object at 0x7f2bdbc67520>, <subprocess.Popen object at 0x7f2bdbc67cd0>, <subprocess.Popen object at 0x7f2bdbbe4070>, <subprocess.Popen object at 0x7f2bdbbe40a0>, <subprocess.Popen object at 0x7f2bdbbe4130>, <subprocess.Popen object at 0x7f2bdbbe4190>, <subprocess.Popen object at 0x7f2bdbbe41f0>, <subprocess.Popen object at 0x7f2bdbbe4880>, <subprocess.Popen object at 0x7f2bdbbe48b0>]

Àß »ì¾Æ ÀÖ´ÂÁö È®ÀÎ... NoneÀ» ¸®ÅÏÇϸé Á×Áö ¾Ê°í »ì¾Ò´Ù´Â ÀǹÌÀÓ.
>>> [x.poll() for x in proc_list]
[None, None, None, None, None, None, None, None, None, None]
>>>

´Ù¸¥ â¿¡¼­ µ¹¾Æ°¡°í ÀÖ´Â Áö È®ÀÎ...
root@ubuntu-20:~/python_subprocess# ps ax | grep ./loop
  6519 pts/2    S      0:15 ./loop
  6520 pts/2    S      0:15 ./loop
  6521 pts/2    S      0:15 ./loop
  6522 pts/2    S      0:15 ./loop
  6523 pts/2    S      0:15 ./loop
  6524 pts/2    R      0:15 ./loop
  6525 pts/2    S      0:15 ./loop
  6526 pts/2    S      0:15 ./loop
  6527 pts/2    S      0:15 ./loop
  6528 pts/2    S      0:15 ./loop

topÀ¸·Î È®ÀÎÇصµ ÀúÁ¤µµ¸é »óÀ§ 10°³¸¦ Â÷ÁöÇÏ°í ÀÖÀ» °Å°í¿ä.
³×ÀÌÃÄ 2020-06
´äº¯µé °¨»çÇÕ´Ï´Ù. Âü°íÇؼ­ °øºÎÇغ¸°Ú½À´Ù~~


QnA
Á¦¸ñPage 4298/5692
2014-05   5017511   Á¤ÀºÁØ1
2015-12   1552912   ¹é¸Þ°¡
2016-03   3598   ¿¹°ü½Å±Ô½Ä
2020-09   3598   Psychophysi¡¦
2018-09   3598   ½Ì¾î¼Û¶óÀÌÅÍ
2020-09   3598   ´ÜÁö
2016-09   3598   ȸ¿øK
2014-03   3598   EYESSHOT
2020-05   3598   °­Çѱ¸
2014-01   3598   ¹Ì¿ì
2018-07   3598   Á¤ÀºÁØ1
2015-06   3598   °×¸Ó
2020-05   3598   ÁØÈ£
2017-04   3598   Goldentooth
2020-05   3597   µðÁöÀ¯
2015-02   3597   Àü¼³¼ÓÀǹ̡¦
2017-04   3597   Çϼ¿È£ÇÁ
2018-04   3597   Sikieiki
2015-01   3597   À̱ԹÎ
2015-03   3597   ¿ÀÁØÈ£5
2018-03   3597   dongcheol
2014-03   3597   ±è½Â±Ç