question

krishnamohan100_189669 avatar image
krishnamohan100_189669 asked krishnamohan100_189669 commented

Issues with kubectl port-forward for week 6 exercise

[FOLLOW UP QUESTION TO #7548]

But then I was not able to execute the urls http://ec2-instance-name:9090 and http://ec2-instance-name:3000 mentioned in Step 4b. Both threw refused connection error. How to address this issue?

I ignored the url error and moved forwarded with Step 5

The next issue which I faced was with port forward.

~/kubernetes-workshop-online/week6-App-in-k8s> kubectl -n my-app port-forward --address 0.0.0.0 pod/astra-backend 9090 &
[1] 2253079
~/kubernetes-workshop-online/week6-App-in-k8s> Forwarding from 0.0.0.0:9090 -> 9090
~/kubernetes-workshop-online/week6-App-in-k8s> Handling connection for 9090
Handling connection for 9090
E0809 14:18:15.369013 2253079 portforward.go:400] an error occurred forwarding 9090 -> 9090: error forwarding port 9090 to pod 4d851590abf69a6832a3e29f56d8fa8ed07bfd001311dd4ea7fa19439d29cdbf, uid : failed to execute portforward in network namespace "/var/run/netns/cni-9fcb4d8f-3b2c-d182-9071-a812c1316d8c": socat command returns error: exit status 1, stderr: "2020/08/09 14:18:15 socat[19005] E connect(5, AF=2 127.0.0.1:9090, 16): Connection refused\n"
E0809 14:18:15.370011 2253079 portforward.go:400] an error occurred forwarding 9090 -> 9090: error forwarding port 9090 to pod 4d851590abf69a6832a3e29f56d8fa8ed07bfd001311dd4ea7fa19439d29cdbf, uid : failed to execute portforward in network namespace "/var/run/netns/cni-9fcb4d8f-3b2c-d182-9071-a812c1316d8c": socat command returns error: exit status 1, stderr: "2020/08/09 14:18:15 socat[19006] E connect(5, AF=2 127.0.0.1:9090, 16): Connection refused\n"

How to resolve this port forward error?

Meanwhile I replayed your Day 6 video for the execution steps and then i changed the port forward statement as

~/kubernetes-workshop-online/week6-App-in-k8s> kubectl -n my-app port-forward pods/astra-backend 9090:9090
After this I did not get the above mentioned errors. I got the message Forwarding from [::1]:9090 -> 9090
But still my url did not work http://<ec2-instance-name>:9090 and http://<ec2-instance-name>:3000
http://<ec2-instance-name>:9090 - threw ERR_EMPTY_RESPONSE error
http://<ec2-instance-name>:3000 - threw Invalid host header

How to resolve the errors?

workshop
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

2 Answers

bettina.swynnerton avatar image
bettina.swynnerton answered krishnamohan100_189669 commented

Hi @krishnamohan100_189669,

I see that you are getting an invalid host header exception when accessing port 3000 on your cloud instance.

This indicates that you are still working with the code base of the UI and backend code before we made some necessary changes to make this work on the cloud instances. One of those changes would have fixed the issue with the invalid host header.

In case we didn't make it clear enough: You will need to update your code on the cloud instance before building the docker images and loading them into kind.

To fix this: Do a git pull on the cloud instance. If you made any changes to files, for example the changes to 05-configMap.yaml, make a backup copy of the file first, then to proceed git stash, git pull and reinstate your modified 05-configMap.yaml

Note: you do not need to delete your kubernetes cluster, and if you have already set up your Cassandra cluster, you can keep it all running. But you will need to rebuild both your backend and ui docker images, load them both into the kind cluster again, delete your backend and ui pods and recreate them with the new images.

Then do a port forward again, and I am confident that this will resolve the issue that you are seeing.

Here is more information about the changes that we made:

https://community.datastax.com/questions/7478/port-5000-blocked-on-cloud-instance-cannot-run-exe.html

I hope this gets you on the right track, let us know if you get stuck.

Thanks!

7 comments Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

krishnamohan100_189669 avatar image krishnamohan100_189669 commented ·

Hi Bettina,

As you had mentioned I did a git stash and git pull. now. Post that I executed all the steps from 1 to 5h but still no luck :(

http://wksg782670.cws-week6.datastaxtraining.com:9090/ and

http://wksg782670.cws-week6.datastaxtraining.com:3000/

are throwing Err_Connection_Refused. I am using Edge browser for the exercise.

Any suggestions?

Regards

Krishna Mohan

0 Likes 0 ·
bettina.swynnerton avatar image bettina.swynnerton ♦♦ krishnamohan100_189669 commented ·

Hi @krishnamohan100_189669,

I checked out your cluster, and it all looks good to me. Both pods are serving locally, you can check this out with for example:

curl http://localhost:3000

and 

curl http://localhost:9090

We get the right response in both cases. Your ports are also already forwarded, the addresses are already in use.

However, we cannot access it from outside of the instance, so it seems that the ports are blocked on the cloud instance, I'll see what else I can find out.

But you followed all steps correctly, the kind cluster is up and running, and your pods serve on the right ports.

0 Likes 0 ·
krishnamohan100_189669 avatar image krishnamohan100_189669 bettina.swynnerton ♦♦ commented ·

Hi Bettina, Thank you for the quick response and the confirmation. Please do let me know once you get the reasons for the issue.

0 Likes 0 ·
Show more comments
Erick Ramirez avatar image
Erick Ramirez answered Erick Ramirez commented

The errors you posted have already been answered previously.

We emailed you last week to advise you of a problem with the cloud instances and we modified the code to workaround the issue. But for it to work, you need to get the latest version of the code.

For details, see Port 5000 blocked on cloud instance, cannot run exercise for week 6. Cheers!

2 comments Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

krishnamohan100_189669 avatar image krishnamohan100_189669 commented ·

Hi Erick

Yes, I am using the latest code. If you notice the error message it is pointing to port 9090 and not 5000.

Regards

Krishna Mohan

0 Likes 0 ·
Erick Ramirez avatar image Erick Ramirez ♦♦ krishnamohan100_189669 commented ·

As Bettina points out in her answer, the "invalid host header" error indicates that you have not deployed the latest code to your k8s cluster. Cheers!

0 Likes 0 ·