Jason Schwarzenberger
|
4e5dc65461
|
don't rescrape if simple.
|
2020-12-10 16:25:51 +13:00 |
|
Jason Schwarzenberger
|
33a25fa34e
|
allow re-scraping if simple scraper was used.
|
2020-12-04 15:34:04 +13:00 |
|
Jason Schwarzenberger
|
da7f6330bf
|
improve meta data scraping.
|
2020-12-04 12:46:46 +13:00 |
|
Jason Schwarzenberger
|
2a2bf4d671
|
add excerpt and scraper details.
|
2020-12-03 16:41:27 +13:00 |
|
Jason
|
afe3e08055
|
etc
|
2020-12-03 01:28:10 +00:00 |
|
Jason Schwarzenberger
|
d1c513b9d6
|
move purify to server side.
|
2020-12-02 15:46:06 +13:00 |
|
Jason Schwarzenberger
|
e53c5fc904
|
fix mistake.
|
2020-12-02 13:28:08 +13:00 |
|
Jason Schwarzenberger
|
59c6f17e67
|
gotta try this on live.
|
2020-12-02 13:22:47 +13:00 |
|
Jason Schwarzenberger
|
f670479bd7
|
progress
|
2020-11-30 18:11:45 +13:00 |
|
Jason
|
085dd47d13
|
fix tvnz time for nzst/nzdt
|
2020-11-26 00:35:32 +00:00 |
|
Jason Schwarzenberger
|
72e2232469
|
fix substack comments.
|
2020-11-26 11:17:25 +13:00 |
|
Jason Schwarzenberger
|
247715a76e
|
adjust feed thread.
|
2020-11-25 12:34:46 +13:00 |
|
Jason Schwarzenberger
|
5c96092a57
|
sort ref_list so newly added is first.
|
2020-11-24 17:28:24 +13:00 |
|
Jason Schwarzenberger
|
bb1413b586
|
sort substack feed by time.
|
2020-11-24 10:56:38 +13:00 |
|
Jason Schwarzenberger
|
fe01ea52e5
|
get favicons for custom substack publications.
|
2020-11-24 10:36:31 +13:00 |
|
Jason Schwarzenberger
|
3daae5fa1b
|
change substack time parsing to misc.time
|
2020-11-23 16:46:54 +13:00 |
|
Jason Schwarzenberger
|
c1b6349771
|
namespace the refs for hn and substack.
|
2020-11-23 16:09:12 +13:00 |
|
Jason
|
54a4c7e55a
|
fix with try-catch
|
2020-11-23 01:20:40 +00:00 |
|
Jason
|
b12a3570b0
|
add logging, extend id length
|
2020-11-21 21:21:31 +00:00 |
|
Jason Schwarzenberger
|
a86eb98c1a
|
fix hn self posts related discussion.
|
2020-11-20 13:06:19 +13:00 |
|
Jason Schwarzenberger
|
abf7f0a802
|
force reader update in update-story.py
|
2020-11-20 12:21:27 +13:00 |
|
Jason Schwarzenberger
|
32bc3b906b
|
add update-story.py
|
2020-11-19 15:06:55 +13:00 |
|
Jason Schwarzenberger
|
f5e65632b8
|
fix comment date.
|
2020-11-19 14:27:24 +13:00 |
|
Jason Schwarzenberger
|
1fe524207e
|
stuff comments.
|
2020-11-19 14:23:01 +13:00 |
|
Jason Schwarzenberger
|
539350a83d
|
port separation.
|
2020-11-18 17:21:37 +13:00 |
|
Jason Schwarzenberger
|
f5b38f5c6b
|
remove readerserver, add declutter.
|
2020-11-18 12:59:35 +13:00 |
|
Jason Schwarzenberger
|
3b885e4327
|
renaming things.
|
2020-11-17 15:54:14 +13:00 |
|
Jason Schwarzenberger
|
5668fa5dbc
|
fix mistake.
|
2020-11-17 12:54:54 +13:00 |
|
Jason Schwarzenberger
|
b771b52501
|
add regex to get a unique ref from each sitemap/category based article url.
|
2020-11-17 12:38:28 +13:00 |
|
Jason Schwarzenberger
|
f5ccd844da
|
fix import error.
|
2020-11-16 15:41:09 +13:00 |
|
Jason Schwarzenberger
|
6a91b9402f
|
split categories, sitemap and other crap out of news.py
|
2020-11-16 15:30:33 +13:00 |
|
Jason Schwarzenberger
|
b23e470317
|
move reddit thresholds as settings variables.
|
2020-11-16 10:11:39 +13:00 |
|
Jason Schwarzenberger
|
7420b5ece9
|
fix microdata multiple authors
|
2020-11-12 17:33:46 +13:00 |
|
Jason Schwarzenberger
|
64ced635cc
|
fix mistake.
|
2020-11-12 17:15:29 +13:00 |
|
Jason Schwarzenberger
|
9318627f1b
|
ability to pass in multiple site maps/category urls.
|
2020-11-12 17:11:51 +13:00 |
|
Jason Schwarzenberger
|
3d0a3f1577
|
support list based json-ld authors.
|
2020-11-12 15:08:23 +13:00 |
|
Jason Schwarzenberger
|
587b10c438
|
recursive sitemaps (sitemap indexes)
|
2020-11-12 14:56:46 +13:00 |
|
Jason
|
00954c6cac
|
local browser scraper
|
2020-11-11 09:26:54 +00:00 |
|
Jason Schwarzenberger
|
3169af3002
|
hostname from settings.
|
2020-11-11 09:46:27 +13:00 |
|
Jason Schwarzenberger
|
d588a60930
|
add source to searchable attributes.
|
2020-11-11 09:37:54 +13:00 |
|
Jason Schwarzenberger
|
408e2870b2
|
tzinfo and microdata schema urls.
|
2020-11-10 16:51:27 +13:00 |
|
Jason Schwarzenberger
|
44b8b36547
|
add data cast in query.
|
2020-11-10 15:50:18 +13:00 |
|
Jason Schwarzenberger
|
1d78b1c592
|
fix favicon url.
|
2020-11-10 15:34:21 +13:00 |
|
Jason Schwarzenberger
|
0374794536
|
Sitemap and Category to get favicon into icon property of story.
|
2020-11-10 15:22:27 +13:00 |
|
Jason Schwarzenberger
|
5efc6ef2d3
|
add related stories (in api only)
|
2020-11-10 14:09:56 +13:00 |
|
Jason Schwarzenberger
|
4ec50e20cb
|
feed thread loop.
|
2020-11-10 10:10:38 +13:00 |
|
Jason Schwarzenberger
|
c1b7877f4b
|
remove limit.
|
2020-11-09 17:54:50 +13:00 |
|
Jason Schwarzenberger
|
7b8cbfc9b9
|
try to make feed only determined by the max age.
|
2020-11-09 17:50:58 +13:00 |
|
Jason Schwarzenberger
|
bfa4108a8e
|
Merge remote-tracking branch 'tanner/master'
|
2020-11-09 16:08:28 +13:00 |
|
Jason Schwarzenberger
|
0bd0d40a31
|
use json type in sqlite.
|
2020-11-09 15:45:10 +13:00 |
|