Issue
I have constructed a news aggregator website which shows 10 headlines with corresponding images, scraped from https://thestar.com ( The Toronto Star ) , using bs4.
However, the major problem is, Everytime i refresh the page, all of the elements dissapear, As seen in the photo below :
Anyone know what the problem is?
Here is the element where my variables are stored:
<div class="col-6">
<center><i><u><h3 class="text-centre">Live News from The Toronto Star</h3></u></i></center>
{% for n, i in toronto %}
<img src="{{i}}">
<strong><h5>{{n}}</h5></strong>
<hr>
{% endfor %}
<br>
</div>
and here’s my views.py
file :
from django.shortcuts import render
import re
import json
import requests
from bs4 import BeautifulSoup
import itertools
# Getting news from The Toronto Star
toi_r = requests.get("https://www.thestar.com/?redirect=true")
toi_soup = BeautifulSoup(toi_r.content, 'html.parser')
toi_headings = toi_soup.find_all('h3')
toi_headings = toi_headings[0:10]# removing footers
toi_news = []
for th in toi_headings:
toi_news.append(th.text)
#Getting news from the NY Times
ht_r = requests.get("https://www.nytimes.com/")
ht_soup = BeautifulSoup(ht_r.content, 'html.parser')
ht_headings = ht_soup.findAll('h3')
ht_headings = ht_headings[0:8] + ht_headings[10:12] + ht_headings[19:21]
ht_news = []
for hth in ht_headings:
ht_news.append(hth.text)
# Getting Images from The Toronto Star
tor_r = requests.get("https://www.thestar.com/")
data = re.search(r"window\.__PRELOADED_STATE__ = (.*)", tor_r.text)
data = json.loads(data.group(1))
def find_images(d):
if isinstance(d, dict):
for k, v in d.items():
if k == "image":
yield v
else:
yield from find_images(v)
if isinstance(d, list):
for v in d:
yield from find_images(v)
images = []
for img in find_images(data):
if img["url"]:
images.append("https://www.thestar.com" + img["url"])
images = images[4:14]
#zip file to concurrently loop news and headlines in the django templates inside my index.html file
toronto = zip(toi_news, images)
def index(req):
return render(req, 'news/index.html', {'toi_news':toi_news, 'ht_news': ht_news, 'images': images, 'toronto': toronto})
Solution
Nevermind, found the solution myself. I just had to add a list()
function around the zip()
function in the toronto
variable on line 79 of my views.py
file. Hope this helped someone out!
Answered By – codingnewbie12345
Answer Checked By – Mildred Charles (AngularFixing Admin)