In-depth analysis of Python dictionary expressions

This article shares with you a detailed explanation of the operation of dictionary expressions in Python. The content is quite good. I hope it can help friends in need.

A Python dictionary expression puzzle

Let’s explore the following obscure Python dictionary expression to find out what happens inside the Python interpreter what happened.

<code class="hljs language-python" style="font-size:.85em;font-family:Consolas, Inconsolata, Courier, monospace;margin:0px .15em;white-space:pre;overflow:auto;border-width:1px;border-style:solid;border-color:rgb(204,204,204);background:rgb(29,31,33);color:rgb(197,200,198);padding:.5em;display:block !important;"><span class="hljs-comment" style="color:rgb(150,152,150);"># 一个python谜题：</span><br><span class="hljs-comment" style="color:rgb(150,152,150);"># 这个表达式计算以后会得到什么结果？</span><br><br><span class="hljs-prompt">>>> </span>{<span class="hljs-keyword" style="color:rgb(178,148,187);">True</span>: <span class="hljs-string" style="color:rgb(181,189,104);">'yes'</span>, <span class="hljs-number" style="color:rgb(222,147,95);">1</span>: <span class="hljs-string" style="color:rgb(181,189,104);">'no'</span>, <span class="hljs-number" style="color:rgb(222,147,95);">1.0</span>: <span class="hljs-string" style="color:rgb(181,189,104);">'maybe'</span>}</code>

Sometimes you come across a very in-depth code example, even just one line of code, but if you think about it enough, it can teach you a lot about a programming language.

Such a code snippet is like a Zen kōan: a question or statement used to question and test the student's progress during the spiritual practice. (Translator's note: Zen kōan, probably a way of spiritual practice, see wikipedia for details)

The code snippet we will discuss is one such example. At first glance, it may look like a simple dictionary expression, but when you think about it carefully, it takes you through a mind-expanding exercise through the cpython interpreter.

I got an inspiration from this short line of code, and once I used it as the content of my speech at a Python conference I attended and started the speech with it.

This has also inspired some positive communication among members of my python mailing list.

So needless to say, this is the code piece. Take a moment to think about the following dictionary expression and what it will compute: #4…

3…

2…

1…

OK, are you okay?

This is the result obtained when the above dictionary expression is evaluated in the cpython interpreter interactive interface:

>>> {True: 'yes', 1: 'no', 1.0: 'maybe'}

I admit that when I first saw this result, I was surprised. But it all makes sense when you look step-by-step at the process that happens.
So, let's think about why we get this - I would say unexpected - result.

Where does this sub-dictionary come from?

When python processes our dictionary expression, it first constructs a new empty dictionary object; then in the order given by the dictionary expression Assign keys and values.

So, when we break it down, our dictionary representation is equivalent to this sequence of statements:

<code class="hljs language-python" style="font-size:.85em;font-family:Consolas, Inconsolata, Courier, monospace;margin:0px .15em;white-space:pre;overflow:auto;border-width:1px;border-style:solid;border-color:rgb(204,204,204);background:rgb(29,31,33);color:rgb(197,200,198);padding:.5em;display:block !important;"><span class="hljs-prompt">>>> </span>{<span class="hljs-keyword" style="color:rgb(178,148,187);">True</span>: <span class="hljs-string" style="color:rgb(181,189,104);">'yes'</span>, <span class="hljs-number" style="color:rgb(222,147,95);">1</span>: <span class="hljs-string" style="color:rgb(181,189,104);">'no'</span>, <span class="hljs-number" style="color:rgb(222,147,95);">1.0</span>: <span class="hljs-string" style="color:rgb(181,189,104);">'maybe'</span>}<br>{<span class="hljs-keyword" style="color:rgb(178,148,187);">True</span>: <span class="hljs-string" style="color:rgb(181,189,104);">'maybe'</span>}</code>

Oddly enough, Python considers all of the statements used in this example Dictionary keys are equal

<code class="hljs language-python" style="font-size:.85em;font-family:Consolas, Inconsolata, Courier, monospace;margin:0px .15em;white-space:pre;overflow:auto;border-width:1px;border-style:solid;border-color:rgb(204,204,204);background:rgb(29,31,33);color:rgb(197,200,198);padding:.5em;display:block !important;"><span class="hljs-prompt">>>> </span>xs = dict()<br><span class="hljs-prompt">>>> </span>xs[<span class="hljs-keyword" style="color:rgb(178,148,187);">True</span>] = <span class="hljs-string" style="color:rgb(181,189,104);">'yes'</span><br><span class="hljs-prompt">>>> </span>xs[<span class="hljs-number" style="color:rgb(222,147,95);">1</span>] = <span class="hljs-string" style="color:rgb(181,189,104);">'no'</span><br><span class="hljs-prompt">>>> </span>xs[<span class="hljs-number" style="color:rgb(222,147,95);">1.0</span>] = <span class="hljs-string" style="color:rgb(181,189,104);">'maybe'</span></code>

OK, but wait here. I'm sure you can accept 1.0 == 1, but why would True be considered equal to 1? This dictionary expression really stumped me the first time I saw it.

After some exploration in the python documentation, I found that python treats bool as a subclass of the int type. This is a snippet in Python 2 and Python 3:

"The Boolean type is a subtype of the integer type, and Boolean values ​​behave like the values ​​0 and 1, respectively, in almost all contexts, the exception being that when converted to a string, the strings 'False' or 'True' are returned, respectively."

"The Boolean type is a subtype of the integer type and is used in almost all contexts Boolean values ​​in the environment behave like the values ​​0 and 1, except that when converted to a string, the string "False" or "True" is returned respectively.

Yes, this means you. You can use bool values ​​as indexes into lists or tuples in Python when programming:

<code class="hljs language-python" style="font-size:.85em;font-family:Consolas, Inconsolata, Courier, monospace;margin:0px .15em;white-space:pre;overflow:auto;border-width:1px;border-style:solid;border-color:rgb(204,204,204);background:rgb(29,31,33);color:rgb(197,200,198);padding:.5em;display:block !important;"><span class="hljs-prompt">>>> </span>[<span class="hljs-string" style="color:rgb(181,189,104);">'no'</span>, <span class="hljs-string" style="color:rgb(181,189,104);">'yes'</span>][<span class="hljs-keyword" style="color:rgb(178,148,187);">True</span>]<br><span class="hljs-string" style="color:rgb(181,189,104);">'yes'</span></code>

但为了代码的可读性起见，您不应该类似这样的来使用布尔变量。（也请建议你的同事别这样做）

Anyway，让我们回过来看我们的字典表达式。

就python而言，True，1和1.0都表示相同的字典键。当解释器计算字典表达式时，它会重复覆盖键True的值。这就解释了为什么最终产生的字典只包含一个键。

在我们继续之前，让我们再回顾一下原始字典表达式：

<code class="hljs language-python" style="font-size:.85em;font-family:Consolas, Inconsolata, Courier, monospace;margin:0px .15em;white-space:pre;overflow:auto;border-width:1px;border-style:solid;border-color:rgb(204,204,204);background:rgb(29,31,33);color:rgb(197,200,198);padding:.5em;display:block !important;"><span class="hljs-prompt">>>> </span>{<span class="hljs-keyword" style="color:rgb(178,148,187);">True</span>: <span class="hljs-string" style="color:rgb(181,189,104);">'yes'</span>, <span class="hljs-number" style="color:rgb(222,147,95);">1</span>: <span class="hljs-string" style="color:rgb(181,189,104);">'no'</span>, <span class="hljs-number" style="color:rgb(222,147,95);">1.0</span>: <span class="hljs-string" style="color:rgb(181,189,104);">'maybe'</span>}<br>{<span class="hljs-keyword" style="color:rgb(178,148,187);">True</span>: <span class="hljs-string" style="color:rgb(181,189,104);">'maybe'</span>}</code>

这里为什么最终得到的结果是以True作为键呢？由于重复的赋值，最后不应该是把键也改为1.0了？

经过对cpython解释器源代码的一些模式研究，我知道了，当一个新的值与字典的键关联的时候，python的字典不会更新键对象本身：

<code class="hljs language-python" style="font-size:.85em;font-family:Consolas, Inconsolata, Courier, monospace;margin:0px .15em;white-space:pre;overflow:auto;border-width:1px;border-style:solid;border-color:rgb(204,204,204);background:rgb(29,31,33);color:rgb(197,200,198);padding:.5em;display:block !important;"><span class="hljs-prompt">>>> </span>ys = {<span class="hljs-number" style="color:rgb(222,147,95);">1.0</span>: <span class="hljs-string" style="color:rgb(181,189,104);">'no'</span>}<br><span class="hljs-prompt">>>> </span>ys[<span class="hljs-keyword" style="color:rgb(178,148,187);">True</span>] = <span class="hljs-string" style="color:rgb(181,189,104);">'yes'</span><br><span class="hljs-prompt">>>> </span>ys<br>{<span class="hljs-number" style="color:rgb(222,147,95);">1.0</span>: <span class="hljs-string" style="color:rgb(181,189,104);">'yes'</span>}</code>

当然这个作为性能优化来说是有意义的 —- 如果键被认为是相同的，那么为什么要花时间更新原来的？

在最开始的例子中，你也可以看到最初的True对象一直都没有被替换。因此，字典的字符串表示仍然打印为以True为键（而不是1或1.0）。

就目前我们所知而言，似乎看起来像是，结果中字典的值一直被覆盖，只是因为他们的键比较后相等。然而，事实上，这个结果也不单单是由__eq__比较后相等就得出的。

等等，那哈希值呢？

python字典类型是由一个哈希表数据结构存储的。当我第一次看到这个令人惊讶的字典表达式时，我的直觉是这个结果与散列冲突有关。

哈希表中键的存储是根据每个键的哈希值的不同，包含在不同的“buckets”中。哈希值是指根据每个字典的键生成的一个固定长度的数字串，用来标识每个不同的键。

这可以实现快速查找。在哈希表中搜索键对应的哈希数字串会快很多，而不是将完整的键对象与所有其他键进行比较，来检查互异性。

然而，通常计算哈希值的方式并不完美。并且，实际上会出现不同的两个或更多个键会生成相同的哈希值，并且它们最后会出现在相同的哈希表中。

如果两个键具有相同的哈希值，那就称为哈希冲突(hash collision)，这是在哈希表插入和查找元素时需要处理的特殊情况。

基于这个结论，哈希值与我们从字典表达中得到的令人意外的结果有很大关系。所以让我们来看看键的哈希值是否也在这里起作用。

我定义了这样一个类来作为我们的测试工具：

<code class="hljs language-python" style="font-size:.85em;font-family:Consolas, Inconsolata, Courier, monospace;margin:0px .15em;white-space:pre;overflow:auto;border-width:1px;border-style:solid;border-color:rgb(204,204,204);background:rgb(29,31,33);color:rgb(197,200,198);padding:.5em;display:block !important;"><span class="hljs-class"><span class="hljs-keyword" style="color:rgb(178,148,187);">class</span> <span class="hljs-title" style="color:rgb(150,152,150);">AlwaysEquals</span>:</span><br>     <span class="hljs-function" style="color:rgb(129,162,190);"><span class="hljs-keyword" style="color:rgb(178,148,187);">def</span> <span class="hljs-title" style="color:rgb(150,152,150);">__eq__</span><span class="hljs-params" style="color:rgb(222,147,95);">(self, other)</span>:</span><br>         <span class="hljs-keyword" style="color:rgb(178,148,187);">return</span> <span class="hljs-keyword" style="color:rgb(178,148,187);">True</span><br><br>     <span class="hljs-function" style="color:rgb(129,162,190);"><span class="hljs-keyword" style="color:rgb(178,148,187);">def</span> <span class="hljs-title" style="color:rgb(150,152,150);">__hash__</span><span class="hljs-params" style="color:rgb(222,147,95);">(self)</span>:</span><br>         <span class="hljs-keyword" style="color:rgb(178,148,187);">return</span> id(self)</code>

这个类有两个特别之处。

第一，因为它的__eq__魔术方法（译者注：双下划线开头双下划线结尾的是一些Python的“魔术”对象）总是返回true，所以这个类的所有实例和其他任何对象都会恒等：

<code class="hljs language-python" style="font-size:.85em;font-family:Consolas, Inconsolata, Courier, monospace;margin:0px .15em;white-space:pre;overflow:auto;border-width:1px;border-style:solid;border-color:rgb(204,204,204);background:rgb(29,31,33);color:rgb(197,200,198);padding:.5em;display:block !important;"><span class="hljs-prompt">>>> </span>AlwaysEquals() == AlwaysEquals()<br><span class="hljs-keyword" style="color:rgb(178,148,187);">True</span><br><span class="hljs-prompt">>>> </span>AlwaysEquals() == <span class="hljs-number" style="color:rgb(222,147,95);">42</span><br><span class="hljs-keyword" style="color:rgb(178,148,187);">True</span><br><span class="hljs-prompt">>>> </span>AlwaysEquals() == <span class="hljs-string" style="color:rgb(181,189,104);">'waaat?'</span><br><span class="hljs-keyword" style="color:rgb(178,148,187);">True</span><br>第二，每个Alwaysequals实例也将返回由内置函数id()生成的唯一哈希值值：<br><br><span class="hljs-prompt">>>> </span>objects = [AlwaysEquals(),<br>               AlwaysEquals(),<br>               AlwaysEquals()]<br><span class="hljs-prompt">>>> </span>[hash(obj) <span class="hljs-keyword" style="color:rgb(178,148,187);">for</span> obj <span class="hljs-keyword" style="color:rgb(178,148,187);">in</span> objects]<br>[<span class="hljs-number" style="color:rgb(222,147,95);">4574298968</span>, <span class="hljs-number" style="color:rgb(222,147,95);">4574287912</span>, <span class="hljs-number" style="color:rgb(222,147,95);">4574287072</span>]</code>

在CPython中，id()函数返回的是一个对象在内存中的地址，并且是确定唯一的。

通过这个类，我们现在可以创建看上去与其他任何对象相同的对象，但它们都具有不同的哈希值。我们就可以通过这个来测试字典的键是否是基于它们的相等性比较结果来覆盖。

正如你所看到的，下面的一个例子中的键不会被覆盖，即使它们总是相等的：

<code class="hljs language-python" style="font-size:.85em;font-family:Consolas, Inconsolata, Courier, monospace;margin:0px .15em;white-space:pre;overflow:auto;border-width:1px;border-style:solid;border-color:rgb(204,204,204);background:rgb(29,31,33);color:rgb(197,200,198);padding:.5em;display:block !important;"><span class="hljs-prompt">>>> </span>{AlwaysEquals(): <span class="hljs-string" style="color:rgb(181,189,104);">'yes'</span>, AlwaysEquals(): <span class="hljs-string" style="color:rgb(181,189,104);">'no'</span>}<br>{ <AlwaysEquals object at <span class="hljs-number" style="color:rgb(222,147,95);">0x110a3c588</span>>: <span class="hljs-string" style="color:rgb(181,189,104);">'yes'</span>,<br>  <AlwaysEquals object at <span class="hljs-number" style="color:rgb(222,147,95);">0x110a3cf98</span>>: <span class="hljs-string" style="color:rgb(181,189,104);">'no'</span> }</code>

下面，我们可以换个思路，如果返回相同的哈希值是不是就会让键被覆盖呢？

<code class="hljs language-python" style="font-size:.85em;font-family:Consolas, Inconsolata, Courier, monospace;margin:0px .15em;white-space:pre;overflow:auto;border-width:1px;border-style:solid;border-color:rgb(204,204,204);background:rgb(29,31,33);color:rgb(197,200,198);padding:.5em;display:block !important;"><span class="hljs-class"><span class="hljs-keyword" style="color:rgb(178,148,187);">class</span> <span class="hljs-title" style="color:rgb(150,152,150);">SameHash</span>:</span><br>    <span class="hljs-function" style="color:rgb(129,162,190);"><span class="hljs-keyword" style="color:rgb(178,148,187);">def</span> <span class="hljs-title" style="color:rgb(150,152,150);">__hash__</span><span class="hljs-params" style="color:rgb(222,147,95);">(self)</span>:</span><br>        <span class="hljs-keyword" style="color:rgb(178,148,187);">return</span> <span class="hljs-number" style="color:rgb(222,147,95);">1</span></code>

这个SameHash类的实例将相互比较一定不相等，但它们会拥有相同的哈希值1：

<code class="hljs language-python" style="font-size:.85em;font-family:Consolas, Inconsolata, Courier, monospace;margin:0px .15em;white-space:pre;overflow:auto;border-width:1px;border-style:solid;border-color:rgb(204,204,204);background:rgb(29,31,33);color:rgb(197,200,198);padding:.5em;display:block !important;"><span class="hljs-prompt">>>> </span>a = SameHash()<br><span class="hljs-prompt">>>> </span>b = SameHash()<br><span class="hljs-prompt">>>> </span>a == b<br><span class="hljs-keyword" style="color:rgb(178,148,187);">False</span><br><span class="hljs-prompt">>>> </span>hash(a), hash(b)<br>(<span class="hljs-number" style="color:rgb(222,147,95);">1</span>, <span class="hljs-number" style="color:rgb(222,147,95);">1</span>)</code>

一起来看看python的字典在我们试图使用SameHash类的实例作为字典键时的结果：

<code class="hljs language-python" style="font-size:.85em;font-family:Consolas, Inconsolata, Courier, monospace;margin:0px .15em;white-space:pre;overflow:auto;border-width:1px;border-style:solid;border-color:rgb(204,204,204);background:rgb(29,31,33);color:rgb(197,200,198);padding:.5em;display:block !important;"><span class="hljs-prompt">>>> </span>{a: <span class="hljs-string" style="color:rgb(181,189,104);">'a'</span>, b: <span class="hljs-string" style="color:rgb(181,189,104);">'b'</span>}<br>{ <SameHash instance at <span class="hljs-number" style="color:rgb(222,147,95);">0x7f7159020cb0</span>>: <span class="hljs-string" style="color:rgb(181,189,104);">'a'</span>,<br>  <SameHash instance at <span class="hljs-number" style="color:rgb(222,147,95);">0x7f7159020cf8</span>>: <span class="hljs-string" style="color:rgb(181,189,104);">'b'</span> }</code>

如本例所示，“键被覆盖”的结果也并不是单独由哈希冲突引起的。

Umm..好吧,可以得到什么结论呢?

Python字典中的键 是否相同（只有相同才会覆盖）取决于两个条件：

1、两者的值是否相等（比较__eq__方法）。
2、比较两者的哈希值是否相同（比较__hash__方法）。

让我们试着总结一下我们研究的结果：

{true：'yes'，1：'no'，1.0：'maybe'}

字典表达式计算结果为 {true：'maybe'}，是因为键true，1和1.0都是相等的，并且它们都有相同的哈希值：

<code class="hljs language-python" style="font-size:.85em;font-family:Consolas, Inconsolata, Courier, monospace;margin:0px .15em;white-space:pre;overflow:auto;border-width:1px;border-style:solid;border-color:rgb(204,204,204);background:rgb(29,31,33);color:rgb(197,200,198);padding:.5em;display:block !important;"><span class="hljs-prompt">>>> </span><span class="hljs-keyword" style="color:rgb(178,148,187);">True</span> == <span class="hljs-number" style="color:rgb(222,147,95);">1</span> == <span class="hljs-number" style="color:rgb(222,147,95);">1.0</span><br><span class="hljs-keyword" style="color:rgb(178,148,187);">True</span><br><span class="hljs-prompt">>>> </span>(hash(<span class="hljs-keyword" style="color:rgb(178,148,187);">True</span>), hash(<span class="hljs-number" style="color:rgb(222,147,95);">1</span>), hash(<span class="hljs-number" style="color:rgb(222,147,95);">1.0</span>))<br>(<span class="hljs-number" style="color:rgb(222,147,95);">1</span>, <span class="hljs-number" style="color:rgb(222,147,95);">1</span>, <span class="hljs-number" style="color:rgb(222,147,95);">1</span>)<br>`</code>

也许并不那么令人惊讶，这就是我们为何得到这个结果作为字典的最终结果的原因：

<code class="hljs language-python" style="font-size:.85em;font-family:Consolas, Inconsolata, Courier, monospace;margin:0px .15em;white-space:pre;overflow:auto;border-width:1px;border-style:solid;border-color:rgb(204,204,204);background:rgb(29,31,33);color:rgb(197,200,198);padding:.5em;display:block !important;"><span class="hljs-prompt">>>> </span>{<span class="hljs-keyword" style="color:rgb(178,148,187);">True</span>: <span class="hljs-string" style="color:rgb(181,189,104);">'yes'</span>, <span class="hljs-number" style="color:rgb(222,147,95);">1</span>: <span class="hljs-string" style="color:rgb(181,189,104);">'no'</span>, <span class="hljs-number" style="color:rgb(222,147,95);">1.0</span>: <span class="hljs-string" style="color:rgb(181,189,104);">'maybe'</span>}<br>{<span class="hljs-keyword" style="color:rgb(178,148,187);">True</span>: <span class="hljs-string" style="color:rgb(181,189,104);">'maybe'</span>}</code>

我们在这里涉及了很多方面内容，而这个特殊的python技巧起初可能有点令人难以置信 —- 所以我一开始就把它比作是Zen kōan。

如果很难理解本文中的内容，请尝试在Python交互环境中逐个去检验一下代码示例。你会收获一些关于python深处知识。

推荐阅读：

• 一个完整的Django入门指南  - 第1部分

• 简单3步，Pycharm 中运行 Django

The above is the detailed content of In-depth analysis of Python dictionary expressions. For more information, please follow other related articles on the PHP Chinese website!