scrapy - python __import__很慢的问题

Question

print int(time.time())
name = "scrapy.extensions.memusage"
__import__(name)
print int(time.time())

执行本段代码耗时30秒，请教可能是什么原因呢？

感谢大神的回答，原因确实是依赖导致的。

我的具体原因是：
hostname, aliases, ipaddrs = gethostbyaddr(name)
这个方法导致的， 跟获取服务器名有关，
解决办法是zai /etc/hosts中加入本机名 解决！

Answer 1

I tested the above code and import scrapy.extensions.memusageboth take around 4s

It should be that scrapy.extensions.memusage has too many dependencies, causing the slow speed

should have nothing to do with __import__

And the official does not recommend using this import module...

If you simply want to import a module (potentially within a package) by name, use importlib.import_module().

Answer 2

The first two people should have answered LZ’s question, but I want to add something

__import__("scrapy.extensions.memusage")
__import__("scrapy")

These two lines are completely equivalent! If you really want to introduce scrapy.extensions.memusage, then you should write like this

__import__("scrapy.extensions.memusage", fromlist=[""])

However, it is of no use. Both of my SSD tests took about 2s.

Answer 3

scrapy.extensions.memusage imports a lot of dependencies by default, which is of course slow. See the number of sys.modules before and after importing this module