Using Python 3.9 and BeautifulSoup to Web Scrape Market, Economic, Financial and Historical Company Earnings Data

January 01, 2021, Added
Added section 5.4 Russell 3000 Index
December 18, 2021, Added
Added section 30 What is JSON
December 11, 2021, Added
Added section 16.4 High Dividend Yield Stocks
September 18, 2021, Update
Python code on how to retrieve historical company earnings
Three standard web technologies

We will use Python and the Python Library BeautifulSoup to retrieve many different types of market, financial and economic data. The types of market, financial and economic data we will retrieve are stock, market index, currency rates, bonds, futures, mutual funds, electronically traded funds (ETF), commodities, balance sheet, income statement, statement of cash flow, GDP, consumer price index, unemployment rate, household income and money supply.

Chapter 28 discusses how to retrieve historical company earnings.

There are two ways to retrieve market data; one of them is web scraping the data; the other is to use the REST web service or REST API that is provided by the website. Most REST API from government agencies are free. When web scraping, we will use the beautifulsoup Python library. The install instruction is located at https://pypi.org/project/beautifulsoup4.
December 18, 2021, Added
Added section 30 What is JSON
December 11, 2021, Added
Added section 16.4 High Dividend Yield Stocks
September 18, 2021, Update
Python code on how to retrieve historical company earnings
Three standard web technologies
January 23, 2021, Update
Added section 3, Standard & Poor’s Sectors and Industries
In this section, we will demonstrate the Yahoo! finance Sector and Industry API. We will discuss how to create the URL for the API.
September 5, 2020, Update
Added section 21, Yahoo! Finance Ticker Info API
In this section, we will demonstrate the Yahoo! finance Ticker Info API. We will discuss how to create the URL for the API.
August 21, 2020, Update
Added 2.3.3 NASDAQ 100
Updated 1.4.5 Yahoo! Finance Python Libraries
August 13, 2020, Update
France, The United States and South Korea
19.1 GDP
19.2 Unemployment Rate
19.3 Consumer Prices
August 11, 2020, Update
Added 5.5 Rate of Return
August 10, 2020, Update
Updated 1.4.1 Visual Studio Code
Added 13.7 API for Financial Statements