virusdefender's blog ʕ•ᴥ•ʔ

最近写后端遇到的几个问题

数据库 id 字段溢出

某个系统,突然数据库无法插入数据,报错如下

1[2017-04-22 23:17:55] - [ERROR] - [utils.api.api:146]  - integer out of range
2Traceback (most recent call last):
3  File "/usr/local/lib/python3.5/dist-packages/django/db/backends/utils.py", line 64, in execute
4    return self.cursor.execute(sql, params)
5psycopg2.DataError: integer out of range

然后表结构是这样的

1user=# select * from acl_ip_data_id_seq;
2   sequence_name    | last_value | start_value | increment_by |      max_value      | min_value | cache_value | log_cnt | is_cycled | is_called 
3--------------------+------------+-------------+--------------+---------------------+-----------+-------------+---------+-----------+-----------
4 acl_ip_data_id_seq | 2147550630 |           1 |            1 | 9223372036854775807 |         1 |           1 |       7 | f         | t
5(1 row)

看到21亿这个数字很熟悉,是 int 最大值,怀疑是数据库 id 字段达到了最大值,无法继续增长了。

这个表是 Django ORM 生成的,看了下 migration,这个字段是 AutoField,发现其实就是 IntegerField

https://docs.djangoproject.com/en/1.9/ref/models/fields/#autofield

但是只有 Django 1.10 才支持更大的整形 ID 字段

https://docs.djangoproject.com/en/1.10/ref/models/fields/#bigautofield

前几天刚看了饿了么的技术一个故障分析,是一样的原因 http://efs.ele.me/?p=246 结果过了几天就遇见了。

localhost 解析出 IPV6 的地址

入口是 Nginx,反代 uwsgi,日志中时不时出现下面的内容

1[error] 54#54: *1644 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.5, server: _, request: "POST /log HTTP/1.1", upstream: "http://[::1]:10000/log", host: "mgt-api"

比较确定的是不是后面的 uwsgi 挂掉了,之后 tcpdump 抓包,发现每次出现问题的都是一个 IPV6 的地址,结合上面的日志恍然大悟,原来是 Nginx 反向代理写的是 localhost 而不是 127.0.0.1,而且在 host 文件中是

1127.0.0.1	localhost
2::1             localhos

gethostbyname 在多个 IP 的情况下,返回结果是随机的,导致有时候被解析到了 IPV6 的地址上了。

Django在 https 下增强的 CSRF 防护

某 Django 系统在 POST 的时候一直提示 CSRF 验证失败,即使看到的 CSRFToken 是没问题的,后来打开 DEBUG 看到提示,原因摘抄 Django 源码。

 1if request.is_secure():
 2    # Suppose user visits http://example.com/
 3    # An active network attacker (man-in-the-middle, MITM) sends a
 4    # POST form that targets https://example.com/detonate-bomb/ and
 5    # submits it via JavaScript.
 6    #
 7    # The attacker will need to provide a CSRF cookie and token, but
 8    # that's no problem for a MITM and the session-independent
 9    # nonce we're using. So the MITM can circumvent the CSRF
10    # protection. This is true for any HTTP connection, but anyone
11    # using HTTPS expects better! For this reason, for
12    # https://example.com/ we need additional protection that treats
13    # http://example.com/ as completely untrusted. Under HTTPS,
14    # Barth et al. found that the Referer header is missing for
15    # same-domain requests in only about 0.2% of cases or less, so
16    # we can use strict Referer checking.
17    referer = force_text(
18        request.META.get('HTTP_REFERER'),
19        strings_only=True,
20        errors='replace'
21    )
22    if referer is None:
23        return self._reject(request, REASON_NO_REFERER)
24
25    referer = urlparse(referer)
26
27    # Make sure we have a valid URL for Referer.
28    if '' in (referer.scheme, referer.netloc):
29        return self._reject(request, REASON_MALFORMED_REFERER)
30
31    # Ensure that our Referer is also secure.
32    if referer.scheme != 'https':
33        return self._reject(request, REASON_INSECURE_REFERER)
34
35    # If there isn't a CSRF_COOKIE_DOMAIN, assume we need an exact
36    # match on host:port. If not, obey the cookie rules.
37    if settings.CSRF_COOKIE_DOMAIN is None:
38        # request.get_host() includes the port.
39        good_referer = request.get_host()
40    else:
41        good_referer = settings.CSRF_COOKIE_DOMAIN
42        server_port = request.get_port()
43        if server_port not in ('443', '80'):
44            good_referer = '%s:%s' % (good_referer, server_port)
45
46    # Here we generate a list of all acceptable HTTP referers,
47    # including the current host since that has been validated
48    # upstream.
49    good_hosts = list(settings.CSRF_TRUSTED_ORIGINS)
50    good_hosts.append(good_referer)
51
52    if not any(is_same_domain(referer.netloc, host) for host in good_hosts):
53        reason = REASON_BAD_REFERER % referer.geturl()
54        return self._reject(request, reason)

提交评论 | 微信打赏 | 转载必须注明原文链接

#Django #后端