FlyerTalk Forums

FlyerTalk Forums (https://www.flyertalk.com/forum/index.php)
-   Mileage Run Discussion (https://www.flyertalk.com/forum/mileage-run-discussion-627/)
-   -   How to best scan this forum without actually reading it? (https://www.flyertalk.com/forum/mileage-run-discussion/1659051-how-best-scan-forum-without-actually-reading.html)

TravelerMSY Feb 28, 2015 1:32 pm

How to best scan this forum without actually reading it?
 
I've been experimenting with Changedetection.com to scan for my desired city pair being mentioned "MSY". It works but gets tripped up sometimes, notably if my username is mentioned because it has MSY in it.

Any better ideas from the computer people out there?

SpammersAreScum Feb 28, 2015 2:34 pm

Well, you could always change your name -- again.:p

The answer would depend on what capabilities changedetection.org provides; I can't seem to find the site to find out. But I found changedetect.com, which offers all of "regular expressions", "ignore tags", and "keep tags"; That should give you the features you need.

iolairemcfadden Feb 28, 2015 3:05 pm

I use yahoo pipes to watch this forum for WAS sales. You can filter the RSS feed by keywords on the title. And also blend multiple feeds into one

TravelerMSY Feb 28, 2015 4:00 pm


I use yahoo pipes to watch this forum for WAS sales. You can filter the RSS feed by keywords on the title. And also blend multiple feeds into one
Please explain.

mikew99 Feb 28, 2015 5:03 pm


Originally Posted by iolairemcfadden (Post 24431612)
I use yahoo pipes to watch this forum for WAS sales. You can filter the RSS feed by keywords on the title. And also blend multiple feeds into one

Great minds think alike! ^


Originally Posted by TravelerMSY (Post 24431776)
Please explain.

I pipe the MR RSS feed through a Yahoo Pipe to filter for my desired cities and airlines and incorporate the result in my home page portal. Yahoo Pipes is basically a high-level programming tool, however, so it's not necessarily easy to use for non-programmers.

Regardless of which tool you use, you can start out with the RSS feed for the Mileage Run Deals forum and go from there: http://www.flyertalk.com/forum/exter...2&forumids=372

iolairemcfadden Mar 1, 2015 12:11 pm


Originally Posted by TravelerMSY (Post 24431776)
Please explain.

I believe this pipe is public:
http://pipes.yahoo.com/pipes/pipe.in...3RGmawspQBJ3AQ <-- non flyertalk pulls in other sources, the pipe it pulls in is another Hopper pipe that parses a Hopper RSS feed. I've not yet figured out how to filter by price so its not so useful since its just a list of destinations and prices from Hopper.
http://pipes.yahoo.com/pipes/pipe.in...2742b511f2bdd9 <-- flyertalk only, filters on some airports and users names, also it looks like this flyertalk pipe might be one that I coned myself

You can view that pipe and edit it if you have a yahoo account. You probably need to clone the pipe, then edit or view source and in a GUI you can edit how its setup. Then you could bookmark the pipe, or pull it into an rss reader via the Get as RSS link when you run the pipe.

Also if you do a search on http://pipes.yahoo.com/ for flyertalk you will find more examples

18sas Mar 2, 2015 9:13 pm

A better question is how to use Yahoo Pipes to gather information from other sites to post here. :)

I'm still at the shallow end of Yahoo Pipes, so is it possible to use some data source other than an RSS feed for a pipe? It would be cool if you could develop a pipe from a kayak or google flights search.

iolairemcfadden Mar 3, 2015 6:46 am


Originally Posted by 18sas (Post 24444181)
I'm still at the shallow end of Yahoo Pipes, so is it possible to use some data source other than an RSS feed for a pipe? It would be cool if you could develop a pipe from a kayak or google flights search.

The closest I have is this Hopper pipe,
http://pipes.yahoo.com/pipes/pipe.in...aedfc84f62690e

Its built on this URL which is flights from WAS to South America, it uses cached data from other people's searches
http://www.hopper.com/flights/feed?u...month=&stay=+7

I think the URL is XML and not RSS so its harder to work with. One of the Hopper guys helped me get it formatted as it is. I tried to parse out the price and didn't have any luck. Ideally you would parse out the price and put in a upper limit - or even better have some other method to determine the distance and combine that with a price. (Honestly I could create an API or some sort to provide distances, or maybe we could pull it from a Google spreadsheet, I have a database of airport distances at home.) I have not spent much time on this.

If you figure out the price please PM me and let me know.

18sas Mar 3, 2015 6:17 pm


Originally Posted by iolairemcfadden (Post 24445823)
The closest I have is this Hopper pipe,
http://pipes.yahoo.com/pipes/pipe.in...aedfc84f62690e

Its built on this URL which is flights from WAS to South America, it uses cached data from other people's searches
http://www.hopper.com/flights/feed?u...month=&stay=+7

I think the URL is XML and not RSS so its harder to work with. One of the Hopper guys helped me get it formatted as it is. I tried to parse out the price and didn't have any luck. Ideally you would parse out the price and put in a upper limit - or even better have some other method to determine the distance and combine that with a price. (Honestly I could create an API or some sort to provide distances, or maybe we could pull it from a Google spreadsheet, I have a database of airport distances at home.) I have not spent much time on this.

If you figure out the price please PM me and let me know.

Thanks. For the price, you can add "&max_price=500" or whatever to the URL string.

Best thing for the distances would be to read them in from the Great Circle Mapper gcmap.com.

iolairemcfadden Mar 27, 2015 7:39 am

This post on frequentmiler discusses another service that allows for filtering, plus SMS notification:
http://frequentmiler.boardingarea.co...ty-or-airline/
For non RSS or less technical users it looks MUCH easier than the yahoo pipes route.

@18sas Thanks I added the filter for &max_price=300


All times are GMT -6. The time now is 12:21 am.


This site is owned, operated, and maintained by MH Sub I, LLC dba Internet Brands. Copyright © 2024 MH Sub I, LLC dba Internet Brands. All rights reserved. Designated trademarks are the property of their respective owners.