Docker之Redis 大量抛Can’t save in background: fork: Out of memory错误的解决方案

评论
8,009 次

摘要

容器中Redis抛Can’t save in background: fork: Out of memory错的解决方法

[[email protected] ~]# docker logs redis_server 
Redis Server Auth Password : ****************
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 3.2.5 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 1
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

1:M 30 Oct 12:01:05.420 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 30 Oct 12:01:05.420 # Server started, Redis version 3.2.5
1:M 30 Oct 12:01:05.420 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1:M 30 Oct 12:01:05.420 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:M 30 Oct 12:01:06.947 * DB loaded from append only file: 1.527 seconds
1:M 30 Oct 12:01:06.947 * The server is now ready to accept connections on port 6379
1:M 30 Oct 12:01:16.074 * 100 changes in 10 seconds. Saving...
1:M 30 Oct 12:01:16.081 * Background saving started by pid 26
26:C 30 Oct 12:01:16.571 * DB saved on disk
26:C 30 Oct 12:01:16.577 * RDB: 8 MB of memory used by copy-on-write
1:M 30 Oct 12:01:16.582 * Background saving terminated with success
1:M 30 Oct 12:01:39.144 * 100 changes in 10 seconds. Saving...
1:M 30 Oct 12:01:39.144 # Can't save in background: fork: Out of memory
1:M 30 Oct 12:01:45.062 * 100 changes in 10 seconds. Saving...
1:M 30 Oct 12:01:45.062 # Can't save in background: fork: Out of memory
1:M 30 Oct 12:01:51.077 * 100 changes in 10 seconds. Saving...
1:M 30 Oct 12:01:51.077 # Can't save in background: fork: Out of memory
1:M 30 Oct 12:01:57.091 * 100 changes in 10 seconds. Saving...
1:M 30 Oct 12:01:57.091 # Can't save in background: fork: Out of memory
1:M 30 Oct 12:02:03.008 * 100 changes in 10 seconds. Saving...
1:M 30 Oct 12:02:03.008 # Can't save in background: fork: Out of memory
1:M 30 Oct 12:02:09.023 * 100 changes in 10 seconds. Saving...
1:M 30 Oct 12:02:09.023 # Can't save in background: fork: Out of memory
1:M 30 Oct 12:02:15.038 * 100 changes in 10 seconds. Saving...
1:M 30 Oct 12:02:15.038 # Can't save in background: fork: Out of memory
1:M 30 Oct 12:02:21.055 * 100 changes in 10 seconds. Saving...
1:M 30 Oct 12:02:21.055 # Can't save in background: fork: Out of memory
1:M 30 Oct 12:02:27.071 * 100 changes in 10 seconds. Saving...
1:M 30 Oct 12:02:27.071 # Can't save in background: fork: Out of memory
1:M 30 Oct 12:02:33.086 * 100 changes in 10 seconds. Saving...
1:M 30 Oct 12:02:33.086 # Can't save in background: fork: Out of memory
1:M 30 Oct 12:02:39.004 * 100 changes in 10 seconds. Saving...
1:M 30 Oct 12:02:39.004 # Can't save in background: fork: Out of memory
1:M 30 Oct 12:02:45.020 * 100 changes in 10 seconds. Saving...
1:M 30 Oct 12:02:45.021 # Can't save in background: fork: Out of memory
1:M 30 Oct 12:02:51.035 * 100 changes in 10 seconds. Saving...
1:M 30 Oct 12:02:51.035 # Can't save in background: fork: Out of memory
1:M 30 Oct 12:02:57.053 * 10 changes in 100 seconds. Saving...
1:M 30 Oct 12:02:57.053 # Can't save in background: fork: Out of memory
1:M 30 Oct 12:03:03.069 * 10 changes in 100 seconds. Saving...
1:M 30 Oct 12:03:03.069 # Can't save in background: fork: Out of memory
1:M 30 Oct 12:03:09.085 * 10 changes in 100 seconds. Saving...
1:M 30 Oct 12:03:09.085 # Can't save in background: fork: Out of memory
1:M 30 Oct 12:03:15.001 * 10 changes in 100 seconds. Saving...
1:M 30 Oct 12:03:15.001 # Can't save in background: fork: Out of memory

由于将线上Redis更换到Docker中跑了，但是今天遇到了一个问题大量抛出Can't save in background: fork: Out of memory错误，GG了下在非容器环境下其实只要该下内核参数就好。
但是我在Docker中测试直接用sysctl 来修改还不行，于是就用了写文件的方法实现