# Self-Hosting #3: "Hello Web, Good bye warranty!" ## RANT It's been a while. I was busy with works in Sharing Life Society, sick with stomach cramp and a bit of cold, and heavily stuck on networking stuff (The solution, like always, is too simple). Man, I almost lost it and wanted to quit so badly. But I didn't, and here is the fruit of my labour. (with a 'u' cause I'm Canadian) Oh yeah, and this is gonna be super long, so if you want the meat of it, I'd read it in the order 3, 5/third try, 2, and 1. ------------ > Following are steps I took in temporal order. The correct order to approach this is vastly different, so don't follow my footstep. ## 1. local ip, remote ip As of now, I've been connecting to the machine using a local ip address.
Your PC, phone, tv and any other IoT devices will have a unique ip address that the router uses to distinguish each machines.
It's what we use to connect local pc to the machine. (using ssh) But to expose the machine to the net, we need to know what the remote/public ip is. For local IP, we used ifconfig, which gives us the local ip address of the machine. However, Your own machine/router cannot know what the remote ip is on its own. It's because we, the client, have no control over the ip address, just as our pc doesn't decide the local IP address.
Fortunately, there are many services that let you check your remote ip. So google "what is my IP" and use any free option.
you could even host your own ip checker! Once you have your public IP, record it somewhere privately. ## 2. port forwarding Now, if you try to access the machine through the remote ip, it will not work. There are few reasons. To name a few: 1. You have the same remote ip for all devices connected to your router. So if you use the remote ip, router will not know where to direct the request to. 2. Which is why most consumer router starts off blocking all requests that are not mandatory. That's where port-forwarding comes to play. Just like how we set the local IP of the tablet static, we: 1. Login to the router, 2. Go the advanced setting, 3. set up port-forwarding (I did it for http:80 and https:443) with the local ip address of the tablet. This opens up port 80 and 443 at the ip of your machine so that people can access the server with http/s request. And now try to start the next web app with port set to 80 and... ``` ~ npm run start > next start -p 80 ⨯ Failed to start server Error: listen EACCES: permission denied 0.0.0.0:80 ``` When we set up ssh in self-hosting #1, we found that termux, without root excess, could not use the default ssh port of 22.
Similarly, We can't use port 80 without root access. All ports below 1024 is locked away from non-root user. So we now need to take the big gun out and root the machine. ## 3. Rooting the SM-P550 (All data will be removed) ### Links - [Magisk for rooting](https://topjohnwu.github.io/Magisk/) - [bifrost for getting the firmware](https://github.com/zacharee/SamloaderKotlin) - [odin for flashing the machine](https://odindownload.com/download/clean/) ### Steps 1. Make sure you have enough space in the machine, about **6 GB of free space**. If you don't, get a **sd card**. Also, make sure **dev mode** is enabled, along with **usb debugging**. 2. Get **Magisk apk** from the link. Make sure to get the version that can support your version of os (27 was the most recent that supported kitkat: Android 7) 3. Get the most recent firmware of your machine from the **bifrost** application. For my case, model number was sm-p550 and region code was XAC (Canada, WIFI-only machine). 4. Copy the **AP** file from the firmware and the **magisk apk** into the machine. 5. Install Magisk from the apk and run it to create modified AP file. (instruction in magisk link) 6. Transfer the new modified AP file back to PC. Use **ADB** or **sd card** to make that transfer. 7. Download **odin**, version 3.13.1, and run it. 8. Reboot the machine in download mode (press and hold home + vol down + power, and when prompted, press vol up) 9. Connect the machine to PC through usb and check that odin detects the machine. If not, install usb driver. Put files from the firmware (step 3) to appropriate slot, replacing the AP with the modified file from the magisk app. 10. Run start and and check that the installation has passed. 11. The machine will reboot and try to update. Do not worry even if it fails midway (32% for me). Wait until the machine goes to **android recovery mode**. 12. use the vol up and down to select factory reset and power to execute. 13. Reboot the system. The machine can stay on the samsung logo for few minutes, so do something else and return to find the machine in initial setup page. 14. Do the initial setups (wifi, data permission for google and samsung, etc) and reinstall **magisk**. 15. Let the magisk do its things, reboot, and now you are rooted. 16. Reinstall **Termux**, set it up, type `su` in terminal. **It will not go through**, but by doing so, Magisk will detect that termux is requesting super user permission. 17. Go to **Magisk**, click on super user tab in bottom nav bar, and give termux super user permission. 18. Go back to termux and type `su`. If you have super user permission, the terminal will change. type `exit` to exit the super user. ### Some key problems I ran into while rooting 1. If you don't have enough storage, Magisk will not be able to create the modified AP file from the original. The factory reset flushes all data anyway, so **backup data** and erase all apps. If that's not enough, use sd card. I used sd card. 2. Canada has quite a few region codes. BMC, TLS, RGS, XAC, etc. If your model is wifi only like mine, XAC should work. For cellular, check with your provider. 3. I started with Odin 3.10.6 to flash the firmware. It resulted in error in the modem section and gave me a **Fail**. Use Odin version > 3.10.6 to prevent such error. If you did face this error, press and hold vol down + power to turn machine off and reboot back into download mode. Then, get a more recent version of odin and repeat the steps. 4. The Magisk instruction talks of **oem unlocking**. Some machines don't have such feature at all, making them naturally oem unlocked. It was the case with sm-p550. So check the **upper left corner** of download mode to see if your machine has oem unlocking. If it does, follow the steps in the Magisk link. 5. Your pc would have stored the machines identity inside a file called `known_hosts` in `$User/.ssh` folder. Remove it so that you can re-connect to the machine through ssh. ## 4. Re-cloning of repo, but with SSH instead of HTTPS I had some problems with storage while rooting the machine. Because of that, I wanted to install as less packages as possible in order to save some space. So when reinstalling git for the web app repo, I didn't want to install anything extra, such as `gh`.
So I decided to use ssh. I mean, I'm already using dropbear for PC connection, so why not for github? ``` ~ pkg install git # this automatically install openssh and deletes dropbear ~ pkg install dropbear # So install dropbear again, which will remove openssh ``` Dropbear, the ssh client/server of my choice, made this a bit complicated. ### first, create keys use `dropbearkey -t ed25519 -f id_ed25519 _C "your email address"` to generate private/public key pair. Copy the content of the public key and use it to create new ssh key in your `github/setting/access/ssh` section.
Move the keys from `home` directory to `.ssh` directory for cleanliness. ### Don't use `ssh`! Of course, I followed the tutorial in the github documentation and I tested the connection with `ssh -T git@github.com`, and it caused error, stating no auth method could be used. Searching high and low, I found [this thread](https://groups.google.com/g/beagleboard/c/h6XiKjT9-ZI/m/xgA0kIGViKgJ), letting me know that using `dbclient`, with options `-y`(accept hostkey) and `-i`(idfile location) will work. and it did. ``` dbclient -yi ~/.ssh/id_ed25519 git@github.com ``` After finding that this worked, I suddenly got curious. > Does chatgpt have the answer? I asked GPT, "how do I access ssh server of github using dropbear?" GPT gave me the steps and I followed. And of course, it was all wrong. But do take this with a grain of salt, as I used the free tier.
Your mileage might vary if you use the paid tier. ### Override the GIT_SSH_COMMAND So can I `git clone` any repo I have on github now? The answer is no. `git clone` still uses `ssh` for connection. So you need to override it using **git config**. 1. create a bash file, with script as below ``` #!/data/data/com.termux/files/usr/bin/bash dbclient -y -i ~/.ssh/id_ed25519 $* ``` 2. In terminal, type `git config --global core.sshCommand ~/path to the bash file` You've successfully overridden the ssh command and now, git clone, pull, push, etc will use dbclient! ## 5. Network mishaps (Iptables, nginx) Wouldn't it be nice if we could just write `npm run start --port 80` and be done with it. But life is constantly giving me lemons. Time to make some lemonades. ### First try: using NAT redirection through iptables I saw that instead of using port 80 directly, we can actually intercept the request to port 80 and redirect it to other ports. With root access, now we can install iptables with the following lines ``` pkg install root-repo // for root related libraries and packages pkg install iptables ``` Why did I install **iptables**? **Iptables** allow user to set chained rules for controlling in/out network traffic.
One can define specific network packet to allow/block, etc.
Basically, it lets you modify the firewall. It is very powerful tool, and with power, comes great complexity. Here's small part of iptables that we will be using. #### Table: Group the rules based on the function 1. Filter: Determine whether a packet can go through the network. It is the default table 2. Network Address Translation (NAT): Reroute packets to different Network/IP/PORT 3. Mangle, Raw and Security: Not used for this application #### Chains: The sequence of packet. Contain list of rules 1. Input: handles incoming packets directed to local application/service. In Filter table 2. Output: Manages packets from local app/service. In all tables 3. Prerouting: Alters packets before any routing decisions are made. in NAT table 4. others: Not used for for this application #### Target/Jump: The response to a rule adherence 1. Accept: Allow packet to pass 2. REDIRECT: Change the destination port Now, without context, most suggestions for NAT redirection goes ``` sudo iptables -A INPUT -i eth0 -p tcp --dport 80 -j ACCEPT sudo iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080 ``` `sudo` for super user permission
The `-t` option let you specify the table, and the lack of it defaults to the filter table.
`-A` is for appending, meaning that the new rule will be the last to be evaluated in the chain,
After `-A`, INPUT and PREROUTING is indicative of which chain the rule appends to.
`-i` is the interface, or what is connected to the network. Your machine's interface is `eth0`
`-p` sets the packet, which in our case is TCP, used for http.
'--dport' is the destination port of the incoming packet. We want incoming packet with port 80.
`-j` let you specify what the iptables should do with a packet that satisfy the rules `-ip --dport`.
`--to-port` redirects the packet to the specified port. Now, in normal system, this should work. But are we working with normal system? No. So I just dropped using iptables and started looking for other solution. ### Second try: NGINX and failure with SE LINUX Saying that NGINX is a pretty popular reverse proxy will be a huge understatement. It's powerful, and just like iptables, it is very confusing. Luckily, There was a template written by **Jakir Hussain**, which I [link right here](https://gist.github.com/iam-hussain/2ecdb934a7362e979e3aa5a92b181153) Here's what I did. ``` ~ pkg install nginx ~ nano ../usr/etc/nginx/nginx.conf # open the editor and replace the server block with what was in the link ~ sudo nginx ~ cd portfolio ~ npm run start ``` This didn't work. I was stumped with error 500 when I tried to access the web app through port 80. (by the way, you don't have to specify port 80) I checked the error.log file in `usr/var/log/nginx` directory.
`socket() (13:permission denied) while connecting to upstream.` with that as the keyword, I searched the google, and the problem seemed to lie in SE Linux. What is SE linux? Well, it's a module called "Security-Enhanced" Linux that makes thing more secure by enforcing some rules. And the solutions on the internet was to relax the rules. The options were: 1. Set SE linux to permissive, by using the command `sudo setenforcing 0` 2. If You can't do that through termux, try a Magisk module that does that in boot. 3. Since not enforcing SE linux is bad for security, instead just bypass what's necessary using `setsebool -P httpd_can_network_connect 1` 4. For more safety, instead of `httpd_can_network_connect`, set `httpd_can_network_relay` to true. Tried all of them. Well, at least tried to try them all because sm-p550 didn't have `setsebool`. Samsung locks SE Linux to "enforcing", even if you are rooted. The command is inside a kernel, where they double check to enforce SE linux. (which is why step 2 didn't work)
In order to find out all of that, I looked at the system log, I looked at the kernel log, I tried to find a config file for SE linux (didn't exist) to change hard-coded way, Tried changing permission of the files and directories in play, etc, etc, etc! Things were out of my hand. I even questioned whether it was the problem with my ISP in some point. I was **FRUSTRATED**. ### Third try: Back to iptables Now, I didn't show you people, but with iptables, you can actually list the rules in a table. `sudo iptables -L`
And you know what? This tablet is filled with rules. So I thought, "Maybe I'll take a look at all the rules there are in this machine. Who knows, maybe something is in there that's causing all of this". ``` ~ touch iptableRules.txt ~ sudo iptables -nL >> ./iptableRules.txt ~ sudo iptables -t nat -nL >> ./iptableRules.txt ~ sudo iptables -t mangle -nL >> ./iptableRules.txt ~ sudo iptables -t raw -nL >> ./iptableRules.txt # back in the PC to copy the txt file to PC > scp -p 8022 /data/data/com.termux/files/home/iptableRules.txt Desktop ``` I couldn't understand more than half of them.
There were custom chains, custom target/jump, A chain calling another chain...
I even forgot to check for another table that exists, called "security" which no one talks about in the internet (unless you specify it) So I said, "-F it"
and I flushed the iptables. ``` ~ sudo iptables -F # Writing the redirection rule again # For NGINX part, I tried with, and without the rules, so don't think that this was why NGINX didn't work ~ sudo iptables -A INPUT -i eth0 -p tcp --dport 80 -j ACCEPT ~ sudo iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080 ``` > And it worked. ---- ## The moral of the story + what's next? There are reason why people use a machine/virtual machine with real OS.
Android and termux come very close, but it's not real. I made it work, but at what cost? Turmoils, times spent, security concerns, etc. But anyway, a W is a W. The next step for this will be to connect this remote ip to a domain, get an ssl certificate so that we can use https, and future-proof by setting up a system for dynamic dns. After that, I think we can focus on security (the minimum) and retrying nginx. Thanks for reading. **BYE!**