SPF Lookup in Go

In this gist, We will check how we can extract SPF records in Go. Prerequisite Go version $ go version go version go1.13 linux/amd64 Dependency DNS Library(https://github.com/miekg/dns) Install dependency $ go get github.com/miekg/dns spfLookup.go Here you can change nameserver according to your requirement. I have specified here google’s name server (8.8.8.8). You can also use cloudflare’s nameserver (1.1.1.1) Conclusion You can make any DNS query with miekg/dns library. In the above script, we have looked up TXT Records and then we have searched for a string containing v=spf1. ...

August 3, 2020 · 1 min · Ashish Tiwari
go ship logs to ELK flow

Shipping Golang logs with ELKB stack

Goal of this blog In this blog, I am going to show you how easily we can write logs to the files in Golang. As well as we are going to store all logs on elasticsearch with EKB (Elasticsearch, Kibana, Beats). Why ELKB stack ? Logs are very important for debugging, reporting, insights etc. In today’s tech world, We uses multiple cloud servers, private servers etc. Which consist of lots of different applications, scripts, programs, daemons, services and they generate their logs too. It is very difficult to go to each server and check all log files in case of debugging or to generate any insights or reporting. ...

June 6, 2020 · 6 min · Ashish Tiwari

Golang basics & Handling 100k hourly webhooks with golang @MimePost

What this talk about? I am working on Golang for the last 1 year from the published date. I have shared some basics of Golang. Also, shared What are the pain points developers face when they migrate from any other language (Especially from web language like PHP) to Golang? I have explained the Webhook architecture of MimePost And how we sending 100k Request hourly( Though Benchmark proves we can scale up to 500k). ...

March 11, 2020 · 1 min · Ashish Tiwari

[SOLVED] Golang fatal error: concurrent map writes

The Problem: Suddenly got below errors which killed my daemon: fatal error: concurrent map writes goroutine 646 [running]: runtime.throw(0x75fd38, 0x15) /usr/local/go/src/runtime/panic.go:774 +0x72 fp=0xc000315e60 sp=0xc000315e30 pc=0x42ecf2 runtime.mapdelete_fast64(0x6f0800, 0xc00008ad50, 0x2b3e) goroutine 1 [sleep]: runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:310 time.Sleep(0x12a05f200) /usr/local/go/src/runtime/time.go:105 +0x157 webhook/worker.Manager() goroutine 6 [IO wait]: internal/poll.runtime_pollWait(0x7fc308de6f08, 0x72, 0x0) /usr/local/go/src/runtime/netpoll.go:184 +0x55 internal/poll.(*pollDesc).wait(0xc000110018, 0x72, 0x0, 0x0, 0x75b00b) /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:92 internal/poll.(*FD).Accept(0xc000110000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) /usr/local/go/src/internal/poll/fd_unix.go:384 +0x1f8 net.(*netFD).accept(0xc000110000, 0xc000050d50, 0xc000046700, 0x7fc308e426d0) /usr/local/go/src/net/fd_unix.go:238 +0x42 net.(*TCPListener).accept(0xc000126000, 0xc000050d80, 0x40dd08, 0x30) /usr/local/go/src/net/tcpsock_posix.go:139 +0x32 net.(*TCPListener).Accept(0xc000126000, 0x72f560, 0xc0000f0180, 0x6f4f20, 0x9c00c0) /usr/local/go/src/net/tcpsock.go:261 +0x47 net/http.(*Server).Serve(0xc0000f4000, 0x7ccbe0, 0xc000126000, 0x0, 0x0) /usr/local/go/src/net/http/server.go:2896 +0x286 net/http.(*Server).ListenAndServe(0xc0000f4000, 0xc0000f4000, 0x8) /usr/local/go/src/net/http/server.go:2825 +0xb7 net/http.ListenAndServe(...) /usr/local/go/src/net/http/server.go:3080 webhook/handler.HandleRequest() Expected behaviour In starting for a few seconds it was working smoothly. ...

February 4, 2020 · 3 min · Ashish Tiwari