Search for a command to run...
Software engineer based in Singapore
Welcome to my website, where I write mostly on tech-related content to share the things I learned from people and from building projects using different tools.
My interests are in tech, traditional finance, cryptocurrency.
No comments yet. Be the first to comment.
Bridging AMD64 and ARM64
Dig into JS bundle to find useful stuff
Tradingview script alert + webhook -> API server -> Telegram
From an almost fully automated pipeline to a fully automated pipeline
Earlier, I discussed how to set up a TradingView webhook and send notifications to telegram for stocks.
The full workflow looks something like this:
Before the market opens, send a telegram notification with stock data from TradingView, VIX Central, and CNN fear and greed index.
When the market closes, receive the TradingView webhook and save the data in Redis.
There is a similar workflow for crypto as well. Before the market opens, a telegram notification is sent with data from CoinMarketCap and the alternative.me fear and greed index. However, there is no TradingView webhook to receive data from.
Note: This post is for educational purpose only.
This is the page on CoinMarketCap where I found the data for the 24-hour price change by sectors.
This page is server-side rendered (SSR), so there is no API request for this data when you inspect the network tab.
CoinMarketCap uses NextJS, which stores the SSR data in window.__NEXT_DATA__, and the data we are interested in is located at props.pageProps.data.
You can also find it in the page source.
Getting the data from HTML is fine, but having an API is better.
In Chrome's source tab, locate the Next.js bundle under the s2.coinmarketcap.com/cmc/_next/static/chunks/pages folder. There, you will find _app**.js and cryptocurrency-category-**.js.
_app**.js contains the application-level code that is bundled for every page.
cryptocurrency-category-**.js contains the code specific to this page.
I still have an older copy of the JavaScript file for this page, which was retrieved sometime last year. The API call is found within a call to F().
getInitialProps is a NextJS lifecycle method that runs when the page is loaded.
Clicking into the function reveals that the endpoint is at /data-api/v3/sector/w/list.
Requesting data from the endpoint gives the outdated version of the top sectors by 24-hour price change, which doesn't match what is shown on the page.
With the updated version of cryptocurrency-category**.js, the API call is at L().
Clicking into the function reveals that the endpoint is /data-api/v4/sector/m/full-list.
Requesting data from the endpoint provides the current list of top sectors by 24-hour price change, which is displayed on the page.
The above 2 requests referenced the javaApi endpoint, which can be found in _app**.js
NextJS knowledge
Since the page is written in NextJS, it's helpful to know where NextJS stores client-side data and understand the lifecycle method for fetching data.
Trial and error
Use the good old debugger to set a breakpoint, reload the page, and check if it gets triggered. If it does, there's a good chance it's the API request we're looking for.
