When SMD access, show 502 Bad Gateway


#1

I am trying to deploy STRATO4.0.0 on the remote machine.
It shows error “502 Bad Gateway” while trying access SMD from client machine’s browser.

Enviroment:

  • CentOS Linux release 7.5.1804 (Core)
  • STRATO4.0.0 on the remote machine
  • Google Chrome 67.0.3396.99(client machine’s browser)

I have tried the following items with no success.

  • ./strato.sh --wipe
  • no set uiPassword, no set NODE_HOST, no set ssl
  • delete all docker image

strato_nginx_1 log:
Ex.

  • 192.AAA.AAA.AAA = client machine’s IP Address
  • 10.BBB.BBB.BBB = remote machine’s external IP address
$ uiPassword=XXXXXX NODE_HOST=10.BBB.BBB.BBB ssl=true ./strato.sh --single
$ docker logs strato_nginx_1
Setting UI password for user "admin"
Adding password for user admin
Waiting for apex to be available...
apex is available
Waiting for vault-wrapper to be available...
vault-wrapper is available
nginx is now running. See the logs below...
2018/09/11 10:39:57 [error] 87#87: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 192.AAA.AAA.AAA, server: , request: "GET /dashboard/ HTTP/1.1", upstream: "http://192.168.48.13:3002/", host: "10.BBB.BBB.BBB"
2018/09/11 10:39:58 [error] 87#87: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 192.AAA.AAA.AAA, server: , request: "GET /dashboard/ HTTP/1.1", upstream: "http://192.168.48.13:3002/", host: "10.BBB.BBB.BBB", referrer: "https://10.BBB.BBB.BBB/dashboard/"

Thanks.


#2

It could be related to your remote computer network settings. May be easier to do an interactive session to gather more debugging information first. Would you be open to talk in near future?


#3

Hi @Cen Sorry. I’m not very used to speaking English.
Is there another way?


#4

@mgmg - you may have an unhealthy docker container. Here’s how to check:

In your terminal, run docker ps. This will list the Docker containers running. If any of the containers show up as ‘Unhealthy’, then docker exec into that container, find the logs, and send them to us.

If you can locate the unhealthy container then we can diagnose the issue.

Alternatively, if all of the containers are healthy, run this command to confirm that you’re pointing to the right instance.

NODE_HOST=<YOUR_IP> ./strato.sh --single

P.S. Our team speaks English, Chinese, Russian, Hindi, French, Dutch, Hebrew, and Spanish. Let me know if we can be helpful!


#5

@eli

All of the containers are healthy.

$ docker ps -a
CONTAINER ID        IMAGE                                                                COMMAND                  CREATED             STATUS                    PORTS                                                          NAMES
58fa2e063c17        registry-aws.blockapps.net:5000/blockapps-repo/nginx:4.0.0           "/docker-run.sh"         18 minutes ago      Up 18 minutes             0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp                       strato_nginx_1
cd411a7dd546        registry-aws.blockapps.net:5000/blockapps-repo/smd:4.0.0             "sh /usr/src/app/doc…"   18 minutes ago      Up 18 minutes             3002/tcp                                                       strato_smd_1
b666986ae11c        registry-aws.blockapps.net:5000/blockapps-repo/dappstore:4.0.0       "sh /usr/src/app/doc…"   18 minutes ago      Up 18 minutes             3000/tcp                                                       strato_dappstore_1
ab8baa56d92e        registry-aws.blockapps.net:5000/blockapps-repo/apex:4.0.0            "sh /usr/src/app/doc…"   18 minutes ago      Up 18 minutes             3001/tcp                                                       strato_apex_1
39829c467ded        registry-aws.blockapps.net:5000/blockapps-repo/postgrest:4.0.0       "sh /doit.sh"            18 minutes ago      Up 18 minutes             3001/tcp                                                       strato_postgrest_1
3a2a371cb994        registry-aws.blockapps.net:5000/blockapps-repo/bloc:4.0.0            "/usr/bin/bloc/doit.…"   18 minutes ago      Up 18 minutes (healthy)   8000/tcp, 8002/tcp                                             strato_bloc_1
fbc3f171b9c2        registry-aws.blockapps.net:5000/blockapps-repo/vault-wrapper:4.0.0   "/doit.sh"               18 minutes ago      Up 18 minutes (healthy)   8000/tcp                                                       strato_vault-wrapper_1
584b9f158f2e        registry-aws.blockapps.net:5000/blockapps-repo/strato:4.0.0          "/var/lib/doit.sh"       18 minutes ago      Up 18 minutes (healthy)   3000/tcp, 0.0.0.0:30303->30303/tcp, 0.0.0.0:30303->30303/udp   strato_strato_1
346506cb7db1        wurstmeister/zookeeper:3.4.6                                         "/bin/sh -c '/usr/sb…"   18 minutes ago      Up 18 minutes             22/tcp, 2181/tcp, 2888/tcp, 3888/tcp                           strato_zookeeper_1
e683e49c0a73        registry-aws.blockapps.net:5000/blockapps-repo/docs:4.0.0            "sh /usr/share/nginx…"   18 minutes ago      Up 18 minutes             8080/tcp                                                       strato_docs_1
2b28e5c8b8af        postgres:9.6                                                         "docker-entrypoint.s…"   18 minutes ago      Up 18 minutes             5432/tcp                                                       strato_postgres_1
40902c95bdf8        wurstmeister/kafka:1.1.0                                             "start-kafka.sh"         18 minutes ago      Up 18 minutes                                                                            strato_kafka_1
b4a2d33bc9d4        redis:3.2                                                            "docker-entrypoint.s…"   18 minutes ago      Up 18 minutes             6379/tcp                                                       strato_redis_1

I think so smd not running.
remote machine’s log:

$ docker logs strato_nginx_1
Using the default password for user "admin"
Waiting for apex to be available...
apex is available
Waiting for vault-wrapper to be available...
vault-wrapper is available
nginx is now running. See the logs below...
2018/09/14 07:45:12 [error] 85#85: *1 connect() failed (111: Connection refused) while connecting to upstream, client: {MY_CLIENT_IP}, server: , request: "GET /dashboard/ HTTP/1.1", upstream: "http://172.24.0.13:3002/", host: "{MY_EXTARNAL_IP}"
2018/09/14 07:45:12 [error] 85#85: *1 connect() failed (111: Connection refused) while connecting to upstream, client: {MY_CLIENT_IP}, server: , request: "GET /dashboard/ HTTP/1.1", upstream: "http://172.24.0.13:3002/", host: "{MY_EXTARNAL_IP}", referrer: "http://{MY_EXTARNAL_IP}/dashboard/"

$ docker inspect strato_nginx_1
...
"NetworkID": "76929c1ee7a6fc7ca495d1d44e80fe08012bc52d2394bbe7153ecab34e70dc9f",
"EndpointID": "1260cdda0dc0dbe0faeadee0410192279990725ecea9351be2854835104fb669",
"Gateway": "172.24.0.1",
"IPAddress": "172.24.0.14",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:ac:18:00:0e",
"DriverOpts": null
...


$ docker inspect strato_smd_1
...
"NetworkID": "76929c1ee7a6fc7ca495d1d44e80fe08012bc52d2394bbe7153ecab34e70dc9f",
"EndpointID": "b02a228e6db7f8b866804959b9e5763f2bde2e11df76571fe0ee60e7db15dad2",
"Gateway": "172.24.0.1",
"IPAddress": "172.24.0.13",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:ac:18:00:0d",
"DriverOpts": null
...


$ docker exec -it strato_nginx_1 curl http://172.24.0.13:3002/
curl: (7) Failed to connect to 172.24.0.13 port 3002: Connection refused

$ docker logs strato_smd_1
+ SINGLE_NODE=true
+ STRATO_GS_MODE=0
+ ssl=false
+ sed -i s|__NODE_NAME__|{MY_EXTARNAL_IP}|g build/index.html
+ sed -i s|__BLOC_URL__|http://{MY_EXTARNAL_IP}/bloc/v2.2|g build/index.html
+ sed -i s|__STRATO_URL__|http://{MY_EXTARNAL_IP}/strato-api/eth/v1.2|g build/index.html
+ sed -i s|__STRATO_DOC_URL__|http://{MY_EXTARNAL_IP}/docs/?url=/strato-api/eth/v1.2/swagger.json|g build/index.html
+ sed -i s|__BLOC_DOC_URL__|http://{MY_EXTARNAL_IP}/docs/?url=/bloc/v2.2/swagger.json|g build/index.html
+ sed -i s|__CIRRUS_URL__|http://{MY_EXTARNAL_IP}/cirrus/search|g build/index.html
+ sed -i s|__APEX_URL__|http://{MY_EXTARNAL_IP}/apex-api|g build/index.html
+ sed -i s|__STRATO_GS_MODE__|0|g build/index.html
+ sed -i s|__SINGLE_NODE__|true|g build/index.html
+ sed -i s|__IS_SSL__|false|g build/index.html
+ sed -i s|__SMD_MODE__|enterprise|g build/index.html
+ sed -i s|__EXT_STORAGE_S3_BUCKET__||g build/index.html
+ sed -i s|__EXT_STORAGE_S3_ACCESS_KEY_ID__||g build/index.html
+ sed -i s|__EXT_STORAGE_S3_SECRET_ACCESS_KEY__||g build/index.html
+ [ 0 != 1 ]
+ ip=1
+ [ 0 = 0 ]
+ ip=1
+ [ 0 = 2 ]
+ [ 0 = 3 ]
+ serve -l 3002 build
+ + curlcurl http://api.mixpanel.com/track/?data=ewogICAgImV2ZW50IjogInNtZF9jb250YWluZXJfc3RhcnRlZCIsCiAgICAicHJvcGVydGllcyI6IHsKICAgICAgICAidG9rZW4iOiAiZGFmMTcxZTkwMzBhYmIzZTMwMmRmOWQ3OGI2YjFhYTAiCiAgICB9Cn0=
 http://api.mixpanel.com/track/?data=ewogICAgImV2ZW50IjogInNtZF9jb250YWluZXJfc3RhcnRlZF9sb2NhbCIsCiAgICAicHJvcGVydGllcyI6IHsKICAgICAgICAidG9rZW4iOiAiZGFmMTcxZTkwMzBhYmIzZTMwMmRmOWQ3OGI2YjFhYTAiCiAgICB9Cn0=
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100     1  100     1    0     0      4      0 --:--:-- --:--:-- --:--:--     4
100     1  100     1    0     0      4      0 --:--:-- --:--:-- --:--:--     4

local machine’s strato smd log:

$ docker logs strato_smd_1

...
+ serve -l 3002 build
   % Total    %  R%e cToetiavle d   %  %X fReercde i vAeverdag e%  SXfpeered d   T imAveer a g e  TSipmeee d       TTiimmee     C uTrirmeen t
      T i m e     C u r  r e n t
                                    Dl o a d  U   p l o a d      DTlootaadl    U pSlpoeandt      T oLteaflt     SSppeeentd
   Lef0t    S p e e0d
100     1  100     1    0     0      5      0 --:--:-- --:--:-- --:--:--     6- : - -    -0 --:---:-:-- --:--:-- ---:--:  --   0:-- --:--:--     0
     6
11UPDATE AVAILABLE The latest version of `serve` is 10.0.0
INFO: Accepting connections at http://localhost:3002

P.S. Our team speaks only Japanese. Sorry.


#6

@mgmg - How about we schedule a time to have a “live chat” conversation. We can type our responses and use an online translator for the language barrier.

This is the link to my calendar to schedule.


#7

@eli - Certainly.
I sent request.


#8

Please tell me what you are doing at smd startup
Similar containers created in the same cloud environment are experiencing similar phenomena
(influence of cloud’s policy?)
There is

  • Added proxy setting to .docker / config.json
  • Added $ HTTP_PROXY to docker-compose file
    I coped with it

Is it possible to deal with strato with a similar solution?


#9

Hi @mgmg, could you run the following commands:
docker exec -it strato_bloc_1 bash
cat logs/bloc


#10

docker logs --tail 100 strato_smd_1


#11

curl http://10.BBB.BBB.BBB:3002


#12

uiPassword=XXXXXX NODE_HOST=10.BBB.BBB.BBB ssl=true ./strato.sh --single


#13

@mgmg Which CentOS version are you using? We were able to deploy successfully on version 7. Can you try to deploy a new node on CentOS 7 and see if it works?


#14

@eli We use centos7.

$ cat /etc/redhat-release
CentOS Linux release 7.5.1804 (Core)

$ uname -a
Linux ******.novalocal 3.10.0-862.el7.x86_64 #1 SMP Fri Apr 20 16:44:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

When I tryed to deploy with new node, but strato don’t worked.


#15

@eli
I solved it!
It was MTU error.

Similar case:
https://www.variantweb.net/blog/docker-mtu-issues-on-openstack/

My Host machine

$ ip a
eth0 mtu 1496

strato_smd_1

$ ip a
eth0 mtu 1500

I added this config to strato docker-compose.yaml.

networks:
  default:
    driver: bridge
    driver_opts:
      com.docker.network.driver.mtu: 1496

thanks for your help!


#16

@mgmg Fantastic! Glad to hear it. Let me know if I can be helpful as you explore our product.

In the meantime, I recommend you try our new Privacy Feature.