it-swarm-ja.com

PM2 godデーモンが存在します(無効なマーク-ヒープ制限に近いコンパクトな割り当てに失敗しました-JavaScriptヒープのメモリ不足)

Pm2を2.10.1から3.5.1に更新しました。 godデーモンプロセスは、古いバージョンと比較して高いメモリ使用量を取得します。また、開始から約15分後に、pm2が次のエラーで突然終了します。

 ==== JS stack trace =========================================

    0: ExitFrame [pc: 0x24b91f2dbe1d]
Security context: 0x065f6f41e6e9 <JSObject>
    1: /* anonymous */(aka /* anonymous */) [0x162edca8f91] [/usr/lib/node_modules/pm2/node_modules/js-git/mixins/fs-db.js:~300] [pc=0x24b91f6da008](this=0x0308a98826f1 <undefined>,data=0x13d779ef8e21 <Object map = 0x2d75412e0a29>)
    2: arguments adaptor frame: 3->1
    3: parseIndex(aka parseIndex) [0x3c7e7b5b4151] [/usr/lib/node_modules/pm2/node_modules/...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x8f9d10 node::Abort() [PM2 v3.5.1: God Daemon (/home/invoker/.pm2)]
 2: 0x8f9d5c  [PM2 v3.5.1: God Daemon (/home/invoker/.pm2)]
 3: 0xaffd0e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [PM2 v3.5.1: God Daemon (/home/invoker/.pm2)]
 4: 0xafff44 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [PM2 v3.5.1: God Daemon (/home/invoker/.pm2)]
 5: 0xef4152  [PM2 v3.5.1: God Daemon (/home/invoker/.pm2)]
 6: 0xef4258 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [PM2 v3.5.1: God Daemon (/home/invoker/.pm2)]
 7: 0xf00332 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [PM2 v3.5.1: God Daemon (/home/invoker/.pm2)]
 8: 0xf00c64 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [PM2 v3.5.1: God Daemon (/home/invoker/.pm2)]
 9: 0xf0215a v8::internal::Heap::CollectAllAvailableGarbage(v8::internal::GarbageCollectionReason) [PM2 v3.5.1: God Daemon (/home/invoker/.pm2)]
10: 0xf0393e v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [PM2 v3.5.1: God Daemon (/home/invoker/.pm2)]
11: 0xecc9c6 v8::internal::Factory::AllocateRawArray(int, v8::internal::PretenureFlag) [PM2 v3.5.1: God Daemon (/home/invoker/.pm2)]
12: 0xecd24a v8::internal::Factory::NewFixedArrayWithFiller(v8::internal::Heap::RootListIndex, int, v8::internal::Object*, v8::internal::PretenureFlag) [PM2 v3.5.1: God Daemon (/home/invoker/.pm2)]
13: 0xecd2f0 v8::internal::Handle<v8::internal::FixedArray> v8::internal::Factory::NewFixedArrayWithMap<v8::internal::FixedArray>(v8::internal::Heap::RootListIndex, int, v8::internal::PretenureFlag) [PM2 v3.5.1: God Daemon (/home/invoker/.pm2)]
14: 0x100c7d9 v8::internal::HashTable<v8::internal::NameDictionary, v8::internal::NameDictionaryShape>::New(v8::internal::Isolate*, int, v8::internal::PretenureFlag, v8::internal::MinimumCapacity) [PM2 v3.5.1: God Daemon (/home/invoker/.pm2)]
15: 0x10316d2 v8::internal::BaseNameDictionary<v8::internal::NameDictionary, v8::internal::NameDictionaryShape>::EnsureCapacity(v8::internal::Handle<v8::internal::NameDictionary>, int) [PM2 v3.5.1: God Daemon (/home/invoker/.pm2)]
16: 0x10318a0 v8::internal::BaseNameDictionary<v8::internal::NameDictionary, v8::internal::NameDictionaryShape>::AddNoUpdateNextEnumerationIndex(v8::internal::Handle<v8::internal::NameDictionary>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyDetails, int*) [PM2 v3.5.1: God Daemon (/home/invoker/.pm2)]
17: 0x1031b4b v8::internal::BaseNameDictionary<v8::internal::NameDictionary, v8::internal::NameDictionaryShape>::Add(v8::internal::Handle<v8::internal::NameDictionary>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyDetails, int*) [PM2 v3.5.1: God Daemon (/home/invoker/.pm2)]
18: 0x1195188 v8::internal::Runtime_AddDictionaryProperty(int, v8::internal::Object**, v8::internal::Isolate*) [PM2 v3.5.1: God Daemon (/home/invoker/.pm2)]
19: 0x24b91f2dbe1d 
2019-07-30T12:35:09: PM2 log: ===============================================================================
2019-07-30T12:35:09: PM2 log: --- New PM2 Daemon started ----------------------------------------------------
2019-07-30T12:35:09: PM2 log: Time                 : Tue Jul 30 2019 12:35:09 GMT+0430 (Iran Daylight Time)
2019-07-30T12:35:09: PM2 log: PM2 version          : 3.5.1
2019-07-30T12:35:09: PM2 log: Node.js version      : 10.16.0
2019-07-30T12:35:09: PM2 log: Current Arch         : x64
2019-07-30T12:35:09: PM2 log: PM2 home             : /home/invoker/.pm2
2019-07-30T12:35:09: PM2 log: PM2 PID file         : /home/invoker/.pm2/pm2.pid
2019-07-30T12:35:09: PM2 log: RPC socket file      : /home/invoker/.pm2/rpc.sock
2019-07-30T12:35:09: PM2 log: BUS socket file      : /home/invoker/.pm2/pub.sock
2019-07-30T12:35:09: PM2 log: Application log path : /home/invoker/.pm2/logs
2019-07-30T12:35:09: PM2 log: Process dump file    : /home/invoker/.pm2/dump.pm2
2019-07-30T12:35:09: PM2 log: Concurrent actions   : 2
2019-07-30T12:35:09: PM2 log: SIGTERM timeout      : 1600
2019-07-30T12:35:09: PM2 log: ===============================================================================

次のコマンドでpm2を起動しています。

pm2 start process.config.js --node-args="--max_new_space_size=4096 --max_executable_size=4096 --max_old_space_size=4096"

そして、process.config.jsには、この形式の約100のプロセスがあります。

 {
  "services": [
    {
      "name": "a",
      "language": "nodejs",
      "replicas": 1,
      "script": "./a/index.js",
      "log_type": "json",
      "exec_mode": "fork",
      "max_memory_restart": "500M",
      "env_production": {
        "NODE_ENV": "PRODUCTION",
        "SERVICE": "a"
      }
    },
    {
      "name": "b",
      "language": "nodejs",
      "replicas": 1,
      "script": "./b/index.js",
      "log_type": "json",
      "exec_mode": "fork",
      "max_memory_restart": "500M",
      "env_production": {
        "NODE_ENV": "PRODUCTION",
        "SERVICE": "b"
      }
    }
  ]
}

古いバージョンのpm2には問題はなく、数日経っても停止することはありませんが、pm2 3.5.1では10〜15分しか実行されません。

1

PM2はvizionを使用し、js-gitの依存関係は大きなgitリポジトリでは正しく機能しません。

モジュールがjs-gitソースに.git/objects/packを読み込もうとしていることがわかります。

  function loadIndex() {
    var indexFile = pathJoin(packDir, "pack-" + packHash + ".idx" );
    fs.readFile(indexFile, function (err, buffer) {
      if (!buffer) return callback(err);
      try {
        cachedIndexes[packHash] = parseIndex(buffer);
      }
      catch (err) { return callback(err); }
      onIndex();
    });
  }

巨大なgitリポジトリがある場合は、ここでOOMを取得するのは簡単です。そのような機能が必要ない場合は、vizionを無効にすることができます(デフォルトではtrueです)。

http://pm2.keymetrics.io/docs/usage/application-declaration/

0
Iaroslav Kolbin