168 Commits

Author	SHA1	Message	Date
Jason Schwarzenberger	a6e1644ddf	Merge remote-tracking branch 'tanner/master'	2020-12-16 11:31:01 +13:00
Jason	5c3b802315	fix server.py	2020-12-15 00:57:20 +00:00
Tanner Collin	c9fb9bd5df	Add Lobsters to feed	2020-12-12 05:26:33 +00:00
Tanner Collin	fd9c9c888d	Update gitignore	2020-12-11 23:49:45 +00:00
Jason Schwarzenberger	66a4953b83	add video max-with	2020-12-10 16:43:33 +13:00
Jason Schwarzenberger	4e5dc65461	don't rescrape if simple.	2020-12-10 16:25:51 +13:00
Jason Schwarzenberger	33a25fa34e	allow re-scraping if simple scraper was used.	2020-12-04 15:34:04 +13:00
Jason Schwarzenberger	da7f6330bf	improve meta data scraping.	2020-12-04 12:46:46 +13:00
Jason Schwarzenberger	2a2bf4d671	add excerpt and scraper details.	2020-12-03 16:41:27 +13:00
Jason	afe3e08055	etc	2020-12-03 01:28:10 +00:00
Jason Schwarzenberger	d1c513b9d6	move purify to server side.	2020-12-02 15:46:06 +13:00
Jason Schwarzenberger	e53c5fc904	fix mistake.	2020-12-02 13:28:08 +13:00
Jason Schwarzenberger	59c6f17e67	gotta try this on live.	2020-12-02 13:22:47 +13:00
Jason Schwarzenberger	f670479bd7	progress	2020-11-30 18:11:45 +13:00
Jason	085dd47d13	fix tvnz time for nzst/nzdt	2020-11-26 00:35:32 +00:00
Jason Schwarzenberger	72e2232469	fix substack comments.	2020-11-26 11:17:25 +13:00
Jason Schwarzenberger	247715a76e	adjust feed thread.	2020-11-25 12:34:46 +13:00
Jason Schwarzenberger	5c96092a57	sort ref_list so newly added is first.	2020-11-24 17:28:24 +13:00
Jason Schwarzenberger	bb1413b586	sort substack feed by time.	2020-11-24 10:56:38 +13:00
Jason Schwarzenberger	fe01ea52e5	get favicons for custom substack publications.	2020-11-24 10:36:31 +13:00
Jason Schwarzenberger	3daae5fa1b	change substack time parsing to misc.time	2020-11-23 16:46:54 +13:00
Jason Schwarzenberger	c1b6349771	namespace the refs for hn and substack.	2020-11-23 16:09:12 +13:00
Jason	54a4c7e55a	fix with try-catch	2020-11-23 01:20:40 +00:00
Jason	b12a3570b0	add logging, extend id length	2020-11-21 21:21:31 +00:00
Jason Schwarzenberger	a86eb98c1a	fix hn self posts related discussion.	2020-11-20 13:06:19 +13:00
Jason Schwarzenberger	abf7f0a802	force reader update in update-story.py	2020-11-20 12:21:27 +13:00
Tanner Collin	42dcf15374	Increase sqlite lock timeout	2020-11-19 21:38:18 +00:00
Tanner Collin	d8a0b77765	Blacklist sec.gov website	2020-11-19 21:37:59 +00:00
Jason Schwarzenberger	32bc3b906b	add update-story.py	2020-11-19 15:06:55 +13:00
Jason Schwarzenberger	f5e65632b8	fix comment date.	2020-11-19 14:27:24 +13:00
Jason Schwarzenberger	1fe524207e	stuff comments.	2020-11-19 14:23:01 +13:00
Jason Schwarzenberger	539350a83d	port separation.	2020-11-18 17:21:37 +13:00
Jason Schwarzenberger	f5b38f5c6b	remove readerserver, add declutter.	2020-11-18 12:59:35 +13:00
Jason Schwarzenberger	3b885e4327	renaming things.	2020-11-17 15:54:14 +13:00
Jason Schwarzenberger	5668fa5dbc	fix mistake.	2020-11-17 12:54:54 +13:00
Jason Schwarzenberger	b771b52501	add regex to get a unique ref from each sitemap/category based article url.	2020-11-17 12:38:28 +13:00
Jason Schwarzenberger	f5ccd844da	fix import error.	2020-11-16 15:41:09 +13:00
Jason Schwarzenberger	6a91b9402f	split categories, sitemap and other crap out of news.py	2020-11-16 15:30:33 +13:00
Jason Schwarzenberger	b23e470317	move reddit thresholds as settings variables.	2020-11-16 10:11:39 +13:00
Jason Schwarzenberger	7420b5ece9	fix microdata multiple authors	2020-11-12 17:33:46 +13:00
Jason Schwarzenberger	64ced635cc	fix mistake.	2020-11-12 17:15:29 +13:00
Jason Schwarzenberger	9318627f1b	ability to pass in multiple site maps/category urls.	2020-11-12 17:11:51 +13:00
Jason Schwarzenberger	3d0a3f1577	support list based json-ld authors.	2020-11-12 15:08:23 +13:00
Jason Schwarzenberger	587b10c438	recursive sitemaps (sitemap indexes)	2020-11-12 14:56:46 +13:00
Jason	00954c6cac	local browser scraper	2020-11-11 09:26:54 +00:00
Jason Schwarzenberger	3169af3002	hostname from settings.	2020-11-11 09:46:27 +13:00
Jason Schwarzenberger	d588a60930	add source to searchable attributes.	2020-11-11 09:37:54 +13:00
Jason Schwarzenberger	408e2870b2	tzinfo and microdata schema urls.	2020-11-10 16:51:27 +13:00
Jason Schwarzenberger	44b8b36547	add data cast in query.	2020-11-10 15:50:18 +13:00
Jason Schwarzenberger	1d78b1c592	fix favicon url.	2020-11-10 15:34:21 +13:00