create account

Scraping Ticker Symbols From Finviz Screener with Selenium in Python by geekgirl

View this thread on: hive.blogpeakd.comecency.com
· @geekgirl ·
$39.16
Scraping Ticker Symbols From Finviz Screener with Selenium in Python
<center>![tickers.png](https://images.hive.blog/DQmTN8ryq6SAHq7gKdgGBdMTFdrH3NkPopc5PM5tM1XwyUo/tickers.png)</center>

If you are a stocks trader, investor, or someone interested in stock market you may have heard or used Finviz. Finviz is a great website that provide many tools. Among which I really like is their stock screener. It provides filters to sort out stocks/companies based on various conditions and criteria. Main categories for filtering are Descriptive, Fundamental, and Technical. 

I have used this screener before to get a sample list of stock tickers for other trading strategy testing tools. I realized my old list became outdated. Over time some stocks get delisted from exchanges, new ones added, some even change ticker symbols. It is good to renew the sample list from time to time. And it should take too much of an effort is this process is automated. 

With a little bit of knowledge of python and selenium this process can be automated. It is also nice to it in actions when browsers opens up and goes through hundreds of webpages in few minutes for work that would otherwise take hours manually. As I was getting ready to update my sample list of tickers, my old script kept giving errors. The issue was that my chromedriver was outdated as well. So, I had to download the latest chrome driver and update the script.

What I am trying to do is, go to Finviz, using their filtering view to get the list of tickers. But I don't want to apply any filters. This time I want to get everything ticker symbol that is available. The page shows me list of 20 companies with ticker, company name, industry, market cap, price, volume etc. While this time I only need tickers, in the future it is good to know that other data is available too and can be obtained with a similar process. 

The page only shows the first twenty companies. To view the rest of them, I would need to click page numbers at the bottom of the page. There are 397 pages total. That would be a lot of repetitive clicking. Luckily, I am automating it and browser will go through the pages automatically.

As the bot goes through the pages it will keep recording the ticker symbols from each page. After its done, it will print them on the screen in a list format. From there I can simply copy/paste and store them in a separate file and use them in other projects. I ended up with a list of 7940 ticker symbols.

Here is the code:

```
import os
from selenium import webdriver

def get_file_path(folder_path,file_name):
    path = os.getcwd() + folder_path
    file_path = path + file_name
    return file_path

def get_page_urls(url):
    driver.get(url)
    pages = driver.find_elements_by_class_name('screener-pages')
    last_page = int(pages[-1].text)
    url_list = [url]
    for i in range(1,last_page):
        text = '&r=' + str(i * 20 +1)
        url_list.append(url+text)
    return url_list

def get_ticker_symbols(url):
    driver.get(url)
    symbols = driver.find_elements_by_class_name('screener-link-primary')
    ticker_list = []
    for sym in symbols:
        ticker_list.append(sym.text)
    return ticker_list

CHROMEDRIVER_PATH = get_file_path('/Desktop/python/','chromedriver')

url = 'https://finviz.com/screener.ashx?v=110'
driver = webdriver.Chrome(executable_path=CHROMEDRIVER_PATH)

url_list = get_page_urls(url)
full_list = []
for url in url_list:
    ticker_list = get_ticker_symbols(url)
    full_list.extend(ticker_list)
    
print(full_list)
print(len(full_list))
driver.quit()
```

In this case, I didn't apply any filters. However, it might be better to apply filters for different situations like creating a stocks watchlist. We can easily apply various filters on Finviz. Then we just need to copy the url and assign it to the `url` variable in the code. This code also requires Selenium to be pip installed on the machine, and latest chromedriver downloaded and the location to the chromedriver specified in the code.


Posted Using [LeoFinance <sup>Beta</sup>](https://leofinance.io/@geekgirl/scraping-ticker-symbols-from-finviz-screener-with-selenium-in-python)
👍  , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , and 204 others
properties (23)
authorgeekgirl
permlinkscraping-ticker-symbols-from-finviz-screener-with-selenium-in-python
categoryhive-167922
json_metadata{"app":"leofinance/0.2","format":"markdown","tags":["python","coding","selenium","stocks","market","neoxian","ctp","palnet","posh","stem","leofinance"],"canonical_url":"https://leofinance.io/@geekgirl/scraping-ticker-symbols-from-finviz-screener-with-selenium-in-python","image":["https://images.hive.blog/DQmTN8ryq6SAHq7gKdgGBdMTFdrH3NkPopc5PM5tM1XwyUo/tickers.png"]}
created2021-03-17 01:39:57
last_update2021-03-17 01:39:57
depth0
children5
last_payout2021-03-24 01:39:57
cashout_time1969-12-31 23:59:59
total_payout_value21.742 HBD
curator_payout_value17.417 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length4,093
author_reputation1,586,488,611,824,452
root_title"Scraping Ticker Symbols From Finviz Screener with Selenium in Python"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id102,415,451
net_rshares57,391,210,565,522
author_curate_reward""
vote details (268)
@cluelessinvestor ·
$0.20
Most of what you are trying to do is way over my head - BUT the website you mentioned was new to me. After spending 5 min on it, I am pretty sure this will be my daily go to along with trading view. Finviz does such a slick job on their visuals - doesnt feel busy but yet pulls me and I start to dig in. Thanks for the new resource!

Posted Using [LeoFinance <sup>Beta</sup>](https://leofinance.io/@cluelessinvestor/re-geekgirl-34dtp8)
👍  
properties (23)
authorcluelessinvestor
permlinkre-geekgirl-34dtp8
categoryhive-167922
json_metadata{"app":"leofinance/0.2","format":"markdown","tags":["hive-167922","leofinance"],"canonical_url":"https://leofinance.io/@cluelessinvestor/re-geekgirl-34dtp8"}
created2021-03-17 13:07:06
last_update2021-03-17 13:07:06
depth1
children0
last_payout2021-03-24 13:07:06
cashout_time1969-12-31 23:59:59
total_payout_value0.100 HBD
curator_payout_value0.100 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length435
author_reputation2,551,203,435,953
root_title"Scraping Ticker Symbols From Finviz Screener with Selenium in Python"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id102,424,357
net_rshares436,937,845,180
author_curate_reward""
vote details (1)
@coininstant ·
$0.29
Oh yeah I love using selenium for my web scraping too!! 
👍  , , , ,
👎  
properties (23)
authorcoininstant
permlinkre-geekgirl-qq3f4s
categoryhive-167922
json_metadata{"tags":["hive-167922"],"app":"peakd/2021.03.7"}
created2021-03-17 03:02:54
last_update2021-03-17 03:02:54
depth1
children2
last_payout2021-03-24 03:02:54
cashout_time1969-12-31 23:59:59
total_payout_value0.143 HBD
curator_payout_value0.143 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length56
author_reputation87,673,095,643,389
root_title"Scraping Ticker Symbols From Finviz Screener with Selenium in Python"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id102,416,570
net_rshares652,301,578,407
author_curate_reward""
vote details (6)
@geekgirl ·
Yes, I like selenium. Is your tool open source?

Off topic: Would you be interesting in voting for leofinance as a witness?
👍  
properties (23)
authorgeekgirl
permlinkqq3hgm
categoryhive-167922
json_metadata{"app":"hiveblog/0.1"}
created2021-03-17 03:53:09
last_update2021-03-17 03:53:09
depth2
children1
last_payout2021-03-24 03:53:09
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length123
author_reputation1,586,488,611,824,452
root_title"Scraping Ticker Symbols From Finviz Screener with Selenium in Python"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id102,417,087
net_rshares7,133,749,934
author_curate_reward""
vote details (1)
@coininstant ·
$0.29
Yeah all the tools I use are open source right off of GitHub, or dockerhub mainly. Sure I will give LeoFinance my vote, thanks!
👍  
👎  
properties (23)
authorcoininstant
permlinkre-geekgirl-qq3iyn
categoryhive-167922
json_metadata{"tags":["hive-167922"],"app":"peakd/2021.03.7"}
created2021-03-17 04:25:36
last_update2021-03-17 04:25:36
depth3
children0
last_payout2021-03-24 04:25:36
cashout_time1969-12-31 23:59:59
total_payout_value0.143 HBD
curator_payout_value0.143 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length127
author_reputation87,673,095,643,389
root_title"Scraping Ticker Symbols From Finviz Screener with Selenium in Python"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id102,417,397
net_rshares642,967,000,978
author_curate_reward""
vote details (2)
@poshbot ·
https://twitter.com/geekjen/status/1372000775069573123
properties (22)
authorposhbot
permlinkre-scraping-ticker-symbols-from-finviz-screener-with-selenium-in-python-20210317t014416z
categoryhive-167922
json_metadata"{"app": "beem/0.24.20"}"
created2021-03-17 01:44:15
last_update2021-03-17 01:44:15
depth1
children0
last_payout2021-03-24 01:44:15
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length54
author_reputation5,554,335,374,496
root_title"Scraping Ticker Symbols From Finviz Screener with Selenium in Python"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id102,415,487
net_rshares0