Project

How to Make an Interactive TCP Server with NodeMCU on the ESP8266

July 22, 2015 by Patrick Lloyd

Learn how to connect your device with Linux Screen using the built-in Lua interpreter to set up the file system and connect to the network. Then, learn how to automate the scripting process with Luatool and put it all together with a basic TCP server that can interact with a pulse-width modulated LED, a potentiometer, and a switch.

Recommended Level

Intermediate

Introduction

This is Part 2 of an advanced series on connecting to and working with the "Internet of Things." Part 1 is located here. This project is relatively straightforward but still involves working intimately with the Linux terminal. Before attempting this project, it's recommended to have some experience with Linux as well as the Lua scripting language. Here is some documentation on Lua  as well as the documentation for the NodeMCU API. Remember, docs are your friend!

Last time, we discussed how to customize and compile the NodeMCU firmware from source using open source tools. Assuming a working NodeMCU installation, we're now going to explore some of the actual functionality of the device and begin interacting with both the "internet" and "things" aspects of IoT. Starting with a fresh flashing of the NodeMCU firmware, this project will walk through connecting to the device with Screen and using the built-in Lua interpreter to set up the file system and connect to the network. We'll then look at automating the scripting process with Luatool and put it all together with a basic TCP server that can interact with a pulse-width modulated LED, a potentiometer, and a switch.

Supplies Needed:

  • Linux Computer (Ubuntu 15.04 Vivid Vervet)
  • NodeMCU Devkit v0.9 with the following user modules compiled: Node, File, GPIO, WiFi, Net, PWM, Timer, ADC, UART
  • WiFi access 
  • 1x LED
  • 1x 100-300 Ohm resistor
  • 1x 5K resistor
  • 1x 10K potentiometer
  • 1x Normally open momentary switch

Software:

  • Git​
  • Python 2.7
  • Pyserial
  • Linux Screen
  • Luatool

​To install these in Ubuntu 15.04, type the following into the terminal:

sudo apt-get install git screen python2.7 python-serial
mkdir -p ~/.opt && cd ~/.opt # just a user-owned directory for software. Replace with what suits your needs
git clone https://github.com/4refr0nt/luatool.git

Connecting to NodeMCU for the First Time

1) Connect to NodeMCU:

All interaction with NodeMCU happens through the serial port at 9600 baud. I prefer to use Screen in the terminal but Minicom or any program that can interact with the serial port will do. Plug in your NodeMCU Devkit and type:

screen /dev/ttyUSB0 9600

If your NodeMCU Devkit happens to live on a different serial port than /dev/ttyUSB0, use dmesg | grep tty to find it.

This should show just a plain black terminal screen with no text. Once connected, press the button labeled USER to reset the device. Or type the following into the terminal and press enter:

node.restart()

 


Press this lil' guy right here

You should now get a bit of gibberish (communication at a different baudrate) and then some kind of welcome message and a '>' prompt. You are now inside NodeMCU's Lua interpreter. The init.lua error is expected since this is a brand new install and we haven't yet sent any scripts to the device. Init.lua is what NodeMCU runs once at startup (sort of like the setup() function in Arduino).

Jacking into the Matrix
The interpreter's inside the computer?! It's so simple!

 

2) Format node file system

If you are working with a fresh install of NodeMCU, we need to format NodeMCU's file system before we can start writing and saving Lua scripts. In the interpreter, type:

file.format()

 

Once completed, you can see information on the file system by using file.fsinfo() call:

remaining, used, total = file.fsinfo()
print("\nFile system info:\nTotal: "..total.." Bytes\nUsed: "..used.." Bytes\nRemaining: "..remaining.." Bytes\n")

 

This will show you total size, free space, and occupied space of just the NodeMCU file system, not the raw memory information of the ESP8266. The ESP8266 has 64 KiB of instruction RAM, 96 KiB of data RAM and about 4 MiB of external flash memory.

Three and a half megabytes? We can work with that.

Note: If you see something like "stdin:1: unexpected symbol near ‘char(27)’” or “stdin: bad header in precompiled chunk” while you're typing commands, don't worry: it's just a little tic of Screen. Hannes Lehmann wrote about the issue on his website saying, "... If you get some errors like “stdin:1: unexpected symbol near ‘char(27)’” or “stdin: bad header in precompiled chunk” then your terminal doesn’t support backspace or arrow input (either copy&paste issue, or you have done a correction of your input). Don’t worry, just repeat the command."

My particular setup with Screen seems like it can handle corrections using backspace but it tends to mess up when I try use the arrow key.

Hello World, Hello WiFi!

3) Connect to WiFi network

Since the main selling point of the ESP8266 is its WiFi stack, let's use the Lua interpreter to connect to the local network and get an IP address. 

The interactive Lua terminal on the device is good for prototyping small bits of code. To connect to your local WiFi and display the IP information, type into the terminal:

wifi.setmode(wifi.STATION)
wifi.sta.config("wifi_name","wifi_pass") -- Replace these two args with your own network
ip, nm, gw=wifi.sta.getip()
print("\nIP Info:\nIP Address: "..ip.." \nNetmask: "..nm.." \nGateway Addr: "..gw.."\n")

 

We're connected!
We're connected!

 

4) Automate with Luatool

Testing small pieces of code with the interpreter is great but what if you want to write something more complicated and have it run automatically at startup? Github user 4refr0nt wrote a program called Luatool that can upload Lua scripts from your computer to the NodeMCU Devkit and save them on the device's file system. Navigate to the Luatool folder that you cloned from Github in the beginning:

cd ~/.opt/luatool/luatool

 

It should have two files in it in addition to luatool.py: init.lua and main.lua. Using your favorite editor, modify the respective files to look like this:


-- init.lua --


-- Global Variables (Modify for your network)
ssid = "my_ssid"
pass = "my_pass"

-- Configure Wireless Internet
print('\nAll About Circuits init.lua\n')
wifi.setmode(wifi.STATION)
print('set mode=STATION (mode='..wifi.getmode()..')\n')
print('MAC Address: ',wifi.sta.getmac())
print('Chip ID: ',node.chipid())
print('Heap Size: ',node.heap(),'\n')
-- wifi config start
wifi.sta.config(ssid,pass)
-- wifi config end

-- Run the main file
dofile("main.lua")

nodemcu2.zip


-- main.lua --


-- Connect 
print('\nAll About Circuits main.lua\n')
tmr.alarm(0, 1000, 1, function()
   if wifi.sta.getip() == nil then
      print("Connecting to AP...\n")
   else
      ip, nm, gw=wifi.sta.getip()
      print("IP Info: \nIP Address: ",ip)
      print("Netmask: ",nm)
      print("Gateway Addr: ",gw,'\n')
      tmr.stop(0)
   end
end)

 -- Start a simple http server
srv=net.createServer(net.TCP)
srv:listen(80,function(conn)
  conn:on("receive",function(conn,payload)
    print(payload)
    conn:send("

Hello, NodeMCU!!!

") end) conn:on("sent",function(conn) conn:close() end) end)

nodemcu3.zip

5) Close your current Screen session (Luatool can't communicate with NodeMCU otherwise) and then upload both files to the NodeMCU: 

python luatool.py --port /dev/ttyUSB0 --src init.lua --dest init.lua --verbose
python luatool.py --port /dev/ttyUSB0 --src main.lua --dest main.lua --verbose

 

6) Reconnect to the NodeMCU Devkit with Screen and press the USER button to reset the device:

screen /dev/ttyUSB0 9600

 

You should then see something like this:

Go to that IP address (in my case 192.168.168.10), and Voila! 


Check it out! We have a teeny, tiny server!

Now for Some Hardware

7) Build the circuit and upload the server code


Here is the schematic. It's fairly simple since we're dealing mostly with software. 

 


The hardware laid out in Fritzing

 

Here's my setup
My own setup

 

Now edit the init.lua and main.lua files from before to look like the following:


-- init.lua --

-- Network Variables
ssid = "your_ssid"
pass = "your_pass"

-- Byline
print('\nAllAboutCircuits.com NodeMCU Example\n')

-- Configure Wireless Internet
wifi.setmode(wifi.STATION)
print('set mode=STATION (mode='..wifi.getmode()..')\n')
print('MAC Address: ',wifi.sta.getmac())
print('Chip ID: ',node.chipid())
print('Heap Size: ',node.heap(),'\n')

-- Configure WiFi
wifi.sta.config(ssid,pass)

dofile("main.lua")

nodemcu4.zip


-- main.lua --

----------------------------------
-- WiFi Connection Verification --
----------------------------------
tmr.alarm(0, 1000, 1, function()
   if wifi.sta.getip() == nil then
      print("Connecting to AP...\n")
   else
      ip, nm, gw=wifi.sta.getip()
      print("IP Info: \nIP Address: ",ip)
      print("Netmask: ",nm)
      print("Gateway Addr: ",gw,'\n')
      tmr.stop(0)
   end
end)


----------------------
-- Global Variables --
----------------------
led_pin = 1
sw_pin = 2
adc_id = 0 -- Not really necessary since there's only 1 ADC...
adc_value = 512

-- Amy from Gargantia on the Verdurous Planet
blink_open = "http://i.imgur.com/kzt3tO8.png"
blink_close = "http://i.imgur.com/KS1dPa7.png"
site_image = blink_open

----------------
-- GPIO Setup --
----------------
print("Setting Up GPIO...")
print("LED")
-- Inable PWM output
pwm.setup(led_pin, 2, 512) -- 2Hz, 50% duty default
pwm.start(led_pin)

print("Switch")
-- Enable input
gpio.mode(sw_pin, gpio.INPUT)

----------------
-- Web Server --
----------------
print("Starting Web Server...")
-- Create a server object with 30 second timeout
srv = net.createServer(net.TCP, 30)

-- server listen on 80, 
-- if data received, print data to console,
-- then serve up a sweet little website
srv:listen(80,function(conn)
	conn:on("receive", function(conn, payload)
		--print(payload) -- Print data from browser to serial terminal
	
		function esp_update()
            mcu_do=string.sub(payload,postparse[2]+1,#payload)
            
            if mcu_do == "Update+LED" then 
            	if gpio.read(sw_pin) == 1 then
            		site_image = blink_open
            		-- Adjust freq
            		pwm.setclock(led_pin, adc_value)
            		print("Set PWM Clock")	
        		elseif gpio.read(sw_pin) == 0 then
        			site_image = blink_close
        			-- Adjust duty cycle
        			pwm.setduty(led_pin, adc_value)
        			print("Set PWM Duty")	
            	end		
            end
            
            if mcu_do == "Read+ADC" then
            	adc_value = adc.read(adc_id)
            	-- Sanitize ADC reading for PWM
				if adc_value > 1023 then
					adc_value = 1023
				elseif adc_value < 0 then
					adc_value = 0
				end
				print("ADC: ", adc_value)
            end
        end

        --parse position POST value from header
        postparse={string.find(payload,"mcu_do=")}
        --If POST value exist, set LED power
        if postparse[2]~=nil then esp_update()end


		-- CREATE WEBSITE --
        
        -- HTML Header Stuff
        conn:send('HTTP/1.1 200 OK\n\n')
        conn:send('\n')
        conn:send('

\n') conn:send('\n') conn:send('ESP8266 Blinker Thing\n') conn:send('

ESP8266 Blinker Thing!

\n') -- Images... just because conn:send('<img src="'..site_image..'" style="height:196px; width:392px" /><br /> <br /> \n') -- Labels conn:send('

ADC Value: '..adc_value..'


') conn:send('

PWM Frequency (Input High): '..adc_value..'Hz

') conn:send('

or

') conn:send('

PWM Duty Cycle (Input Low): '..(adc_value * 100 / 1024)..'%


') -- Buttons conn:send('

\n') conn:send('\n') conn:send('\n') conn:send('\n') conn:on("sent", function(conn) conn:close() end) end) end)

How_to_Make_an_Interactive_TCP_Server_with_NodeMCU_on_the_ESP8266.zip

Again, close any active Screen sessions for Luatool and upload both files to the NodeMCU: 

python luatool.py --port /dev/ttyUSB0 --src init.lua --dest init.lua --verbose
python luatool.py --port /dev/ttyUSB0 --src main.lua --dest main.lua --verbose

 

8) Reconnect to the NodeMCU Devkit with Screen and press the USER button to reset the device:

screen /dev/ttyUSB0 9600

 

Here is a video of the project in action:

What's Going On Here?

When the user presses the "Read ADC" button in the browser, the browser is updated with the current ADC reading of the potentiometer and that value is posted to the NodeMCU's serial terminal if you have it open. If the pushbutton is not pressed, the input pin is pulled high which means that the current ADC reading will be used to set the LED's PWM frequency. If it is pressed, and the input pin is pulled low, the LED's PWM duty cycle is adjusted. You also get a different image in the browser depending on what LED parameter is being set.

Now let's take a moment to dig through the code to see how this is implemented. Init.lua contains mostly code from the "Hello Word, Hello WiFi" section. It displays some information about the chip and connects to the wireless network. Main.lua is where all the fun happens -- it's a modification of the code here. The flow of that script is to print IP information, initialize some global variables, configure the I/O, and then create a TCP server that listens on port 80. Any time a button is pressed, an HTTP POST method is called by the browser. The string.find() method in the script looks though the HTTP header and tries to find any mention of a button named "mcu_do". If this does turn up, the esp_update() function is called and depending on the value assigned to mcu_do, it will either read the ADC or update the parameters of the LED. And there you have it, a bit of hardware that can interact with a browser in a meaningful way and vice versa.

 

Closing Remarks

This project only scratches the surface of what you can do with the ESP8266 and NodeMCU. It can act as an MQTT broker, talk UDP as well as TCP, perform cryptography, communicate with peripherals over I2C and SPI, and a ton more. The NodeMCU Devkit is a powerful hardware device that can enable very powerful IoT applications but is by no means the only or even the best solution for all projects. Keep your eyes peeled as new solutions in software and hardware spring into the blossoming IoT landscape.

 

Give this project a try for yourself! Get the BOM.

11 Comments
  • X
    xtal September 07, 2015

    I downloaded the interactive web server with the blink picture - it works fine IF I use a wired connection to router and
    wireless to ESP8266,,,However if I use wireless to router and wireless to ESP8266 I get double displays ,,,maybe I’m getting an error and seeing a retransmit , if so how do I stop the double display , or get it to reposition correctly….........click read ADC several times to see problem

    Like. Reply
  • E
    edmondkreuk December 26, 2015

    All I get is a white page in my browser… At first I edited main.lua and init.lua myself, but I also did a copy-paste. Same result. In Screen I do see the text, but the “Hello World” doesn’t appear in my browser. What went wrong?

    Like. Reply
  • Marcel Stör January 11, 2016

    It was kind of a bug this ever worked. Due to the event-driven asynchronous nature of NodeMCU multiple consecutive calls to `conn:send` are not guaranteed to be executed in the order you define them. See https://github.com/nodemcu/nodemcu-firmware/issues/730#issuecomment-154241161 for more details.

    Like. Reply