Flightplan: How to search a year of award inventory [no longer maintained]
#137
Original Poster
Join Date: Jan 2010
Posts: 189
I've found KE usually to be pretty fast (maybe they're having a bad day, or purposefully throttling you), but you can change the timeout directly in the code here:
https://github.com/flightplan-tool/f...engine.js#L177
One of the changes in V2 was to better consolidate configuration in one place (like timeouts) but it still requires editing a file. Not sure if it's really worthwhile to expose settings like timeout's in the command line interface. There's a tradeoff, where you don't want to make the timeout too high, because sometimes it's better to just give up sooner and retry the request (that may be faster than waiting for the original request to complete). SQ will even hang search requests indefinitely, if they detect you're a bot.
https://github.com/flightplan-tool/f...engine.js#L177
One of the changes in V2 was to better consolidate configuration in one place (like timeouts) but it still requires editing a file. Not sure if it's really worthwhile to expose settings like timeout's in the command line interface. There's a tradeoff, where you don't want to make the timeout too high, because sometimes it's better to just give up sooner and retry the request (that may be faster than waiting for the original request to complete). SQ will even hang search requests indefinitely, if they detect you're a bot.
Last edited by jd20; Sep 19, 2018 at 12:39 pm Reason: Adding link
#140
Join Date: Aug 2010
Programs: QF WP
Posts: 42
Install on fresh ubuntu 18.04 and seems commander has an issue related to this: https://github.com/tj/commander.js/issues/786
andrew@FlightPlanVM2:~$ flightplan search
flightplan-search(1) does not exist, try --help
andrew@FlightPlanVM2:~$
andrew@FlightPlanVM2:~$ flightplan search
flightplan-search(1) does not exist, try --help
andrew@FlightPlanVM2:~$
#141
Join Date: Aug 2010
Programs: QF WP
Posts: 42
Ended up just calling node directly which works, but SQ saying 403, yet logging in manually works.
andrew@fp:~$ node .config/yarn/global/node_modules/flightplan-tool/bin/cli-search.js -w SQ -f MEL -t SIN -c first -s 2018-11-01 -e 2018-11-30 -q 2 -h
Searching 30 days of award inventory: 11/01/2018 - 11/30/2018
SQ: Logging in...
SQ: Login success!
SQ: Loading airports...
SQ: Found 687 airports
SQ: DEPARTURE [SIN -> MEL] - 11/01/2018 (2 Passengers)
Received non-OK HTTP Status Code: 403
SQ: DEPARTURE [SIN -> MEL] - 11/02/2018 (2 Passengers)
SQ: Cool-down period, resuming in 13 minutes
andrew@fp:~$ node .config/yarn/global/node_modules/flightplan-tool/bin/cli-search.js -w SQ -f MEL -t SIN -c first -s 2018-11-01 -e 2018-11-30 -q 2 -h
Searching 30 days of award inventory: 11/01/2018 - 11/30/2018
SQ: Logging in...
SQ: Login success!
SQ: Loading airports...
SQ: Found 687 airports
SQ: DEPARTURE [SIN -> MEL] - 11/01/2018 (2 Passengers)
Received non-OK HTTP Status Code: 403
SQ: DEPARTURE [SIN -> MEL] - 11/02/2018 (2 Passengers)
SQ: Cool-down period, resuming in 13 minutes
#142
Join Date: Nov 2010
Location: Sydney
Programs: NH AMC, EY Guest Silver,MPC GO
Posts: 210
#143
Join Date: Aug 2010
Programs: QF WP
Posts: 42
https://github.com/flightplan-tool/flightplan/issues/7
#144
Original Poster
Join Date: Jan 2010
Posts: 189
New version released: 0.2
Hi all, Happy Columbus Day!
The new version of Flightplan is out, you can get it by opening a command line and running:
After that (it may take a while to install), run "flightplan --version", and you should see the following:
Important Instructions:
Lots of new stuff in this release, including:
SFO-HKG award availability from six different engines.
Awards included from multiple partner airlines.
At a glance view of segment details, award pricing, and quantities!
Same UA award, fetched from both NH and AC.
Support for mixed cabin awards.
The new version of Flightplan is out, you can get it by opening a command line and running:
yarn global upgrade flightplan-tool
❯ flightplan --version
0.2.2
0.2.2
- The database format has changed significantly, if running flightplan from a directory with an existing database, please delete the "db" directory. Otherwise, you will get an error when running flightplan. You may also delete the "data" directory to clean things up, this is optional.
- The JSON format for storing usernames / passwords has been dropped in favor of a simpler CSV-like format. When you run flightplan, it will ask if you'd like it to convert the old JSON file. You'll still need to add entries for AC and BA manually (since they're new). In whatever directory you run "flightplan" command from, look for "config/accounts.txt" and open it with a text editor (like notepad.exe on Windows, or TextEdit on the Mac). The format is pretty self-explanatory (e.g. "BA:Username:Password").
Lots of new stuff in this release, including:
- Support for two new engines: AC and BA
- Partner award search (Some engines like AC automatically return partner awards, whereas others like KE or SQ you must explicitly request it by adding --partners to the search command)
- Multi-segment awards are now supported (before only non-stop awards were considered)
- Your search settings in the web UI are now remembered, even after closing the browser.
- When you hover over a date in the web UI, you will still see an overview of award availability like before. If you click on the date, it will pop up a full schedule of all awards, including detailed segment information (such as cabin, aircraft, connection times, lag days, durations).
- When available, award cost in miles is also shown, in the detailed view (in a few cases it's unavailable, like BA awards or KE partner awards).
- There are some additional filtering controls in the web UI: mixed cabin awards, partner awards, # of stops. Additional things like red-eye flights, duration, award pricing are on the todo list.
- Proxy support, in case your IP is being blocked. (Currently at the API level, adding a command-line switch is on the todo list.)
- General reliability / compatibility fixes, any engines that were obviously broken, have been fixed. Some other issues, like certain airport codes with NH not being selectable, have also been fixed.
- Due to all the new stuff, I expect there to be bugs. I've not had time to really stress test the new version, or check many of the edge cases, so if you see weirdness or errors, please report it! A Github issue is the best way, as it keeps the conversation easier to follow, but if you don't have a Github account, posting in this thread is fine too.
- Support for more websites is coming, but I'm more focused right now on ironing out the bugs in this new release. If there's a website you'd really like to see added, be sure to mention it in the thread!
SFO-HKG award availability from six different engines.
Awards included from multiple partner airlines.
At a glance view of segment details, award pricing, and quantities!
Same UA award, fetched from both NH and AC.
Support for mixed cabin awards.
Last edited by jd20; Oct 10, 2018 at 12:31 pm Reason: Update instructions to include deleting old database
#145
Join Date: Feb 2004
Location: New York
Programs: DL Plat, AA EXP, UA 1K, AMEX Cent
Posts: 684
Having an issue on new version. Have removed and reinstalled several times. Getting this message when running any search (search in bold is an example)
PS C:\flightplan> flightplan search -w CX -f JFK -t HKG -c first -s 2018-10-11 -e 2018-10-15 -o -h
flightplan search 0.2.2
Searching 5 days of award inventory: 2018-10-11 - 2018-10-15
error A fatal error occurred!
SqliteError: no such table: requests
at requests (C:\Users\XXXXX\AppData\Local\Yarn\Data\global\nod e_modules\flightplan-tool\shared\routes.js:86:21)
at Object.find (C:\Users\XXXXX\AppData\Local\Yarn\Data\global\nod e_modules\flightplan-tool\shared\routes.js:38:21)
at redundant (C:\Users\XXXXX\AppData\Local\Yarn\Data\global\nod e_modules\flightplan-tool\bin\cli-search.js:220:28)
at main (C:\Users\XXXXX\AppData\Local\Yarn\Data\global\nod e_modules\flightplan-tool\bin\cli-search.js:283:27)
at Object.<anonymous> (C:\Users\XXXXX\AppData\Local\Yarn\Data\global\nod e_modules\flightplan-tool\bin\cli-search.js:342:1)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
PS C:\flightplan> flightplan search -w CX -f JFK -t HKG -c first -s 2018-10-11 -e 2018-10-15 -o -h
flightplan search 0.2.2
Searching 5 days of award inventory: 2018-10-11 - 2018-10-15
error A fatal error occurred!
SqliteError: no such table: requests
at requests (C:\Users\XXXXX\AppData\Local\Yarn\Data\global\nod e_modules\flightplan-tool\shared\routes.js:86:21)
at Object.find (C:\Users\XXXXX\AppData\Local\Yarn\Data\global\nod e_modules\flightplan-tool\shared\routes.js:38:21)
at redundant (C:\Users\XXXXX\AppData\Local\Yarn\Data\global\nod e_modules\flightplan-tool\bin\cli-search.js:220:28)
at main (C:\Users\XXXXX\AppData\Local\Yarn\Data\global\nod e_modules\flightplan-tool\bin\cli-search.js:283:27)
at Object.<anonymous> (C:\Users\XXXXX\AppData\Local\Yarn\Data\global\nod e_modules\flightplan-tool\bin\cli-search.js:342:1)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
#146
Join Date: Feb 2013
Location: DTW
Programs: DL Diamond/AA Platinum, Hyatt/Hilton Diamond
Posts: 245
Hello! The demo is impressive and love what you did on visualization, great job!!
I want to try it out myself but could not get it to work on either windows10 or maxos mohave.
Windows 10:
node & yarn installed successfully
upon executing 'yarn global add flightplan-tool', it throws an error on 'better-sqlite3'
I tried on MacOS as well, this time filghtplan installed successfully
but upon running, it gives a different error:
Really wanted to make this work and can't figure out how. If you could help that would be great! Thanks!
I want to try it out myself but could not get it to work on either windows10 or maxos mohave.
Windows 10:
node & yarn installed successfully
upon executing 'yarn global add flightplan-tool', it throws an error on 'better-sqlite3'
C:\Users\Jerry>yarn global add flightplan-tool
yarn global v1.10.1
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
[4/5] ⠄ better-sqlite3
[2/5] ⠄ puppeteer
[-/5] ⠄ waiting...
[-/5] ⠄ waiting...
error C:\Users\Jerry\AppData\Local\Yarn\Data\global\node _modules\better-sqlite3: Command failed.
Exit code: 1
Command: node-gyp rebuild
Arguments:
Directory: C:\Users\Jerry\AppData\Local\Yarn\Data\global\node _modules\better-sqlite3
Output:
C:\Users\Jerry\AppData\Local\Yarn\Data\global\node _modules\better-sqlite3>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modules\npm\bin\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild ) else (node "" rebuild )
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] win32 x64
gyp info spawn C:\Users\Jerry\.windows-build-tools\python27\python.EXE
gyp info spawn args [ 'C:\\Program Files\\nodejs\\node_modules\\npm\\node_modules\\no de-gyp\\gyp\\gyp_main.py',
gyp info spawn args 'binding.gyp',
gyp info spawn args '-f',
gyp info spawn args 'msvs',
gyp info spawn args '-G',
gyp info spawn args 'msvs_version=2015',
gyp info spawn args '-I',
gyp info spawn args 'C:\\Users\\Jerry\\AppData\\Local\\Yarn\\Data\\glo bal\\node_modules\\better-sqlite3\\build\\config.gypi',
gyp info spawn args '-I',
gyp info spawn args 'C:\\Program Files\\nodejs\\node_modules\\npm\\node_modules\\no de-gyp\\addon.gypi',
gyp info spawn args '-I',
gyp info spawn args 'C:\\Users\\Jerry\\.node-gyp\\8.12.0\\include\\node\\common.gypi',
gyp info spawn args '-Dlibrary=shared_library',
gyp info spawn args '-Dvisibility=default',
gyp info spawn args '-Dnode_root_dir=C:\\Users\\Jerry\\.node-gyp\\8.12.0',
gyp info spawn args '-Dnode_gyp_dir=C:\\Program Files\\nodejs\\node_modules\\npm\\node_modules\\no de-gyp',
gyp info spawn args '-Dnode_lib_file=C:\\Users\\Jerry\\.node-gyp\\8.12.0\\<(target_arch)\\node.lib',
gyp info spawn args '-Dmodule_root_dir=C:\\Users\\Jerry\\AppData\\Local\ \Yarn\\Data\\global\\node_modules\\better-sqlite3',
gyp info spawn args '-Dnode_engine=v8',
gyp info spawn args '--depth=.',
gyp info spawn args '--no-parallel',
gyp info spawn args '--generator-output',
gyp info spawn args 'C:\\Users\\Jerry\\AppData\\Local\\Yarn\\Data\\glo bal\\node_modules\\better-sqlite3\\build',
gyp info spawn args '-Goutput_dir=.' ]
gyp info spawn C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\15.0\Bin\MSBuild.ex e
gyp info spawn args [ 'build/binding.sln',
gyp info spawn args '/clp:Verbosity=minimal',
gyp info spawn args '/nologo',
gyp info spawn args '/p:Configuration=Release;Platform=x64' ]
Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\Common7\IDE\VC\VCTargets\Mi crosoft.Cpp.Platform.targets(67,5): error MSB8020: The build tools for v140 (Platform Toolset = 'v140') cannot be found. To build using the v140 build tools, please install v140 build tools. Alternatively, you may upgrade to the current Visual Studio tools by selecting the Project menu or right-click the solution, and then selecting "Retarget solution". [C:\Users\Jerry\AppData\Local\Yarn\Data\global\node _modules\better-sqlite3\build\deps\action_before_build.vcxproj]
gyp ERR! build error
gyp ERR! stack Error: `C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\15.0\Bin\MSBuild.ex e` failed with exit code: 1
gyp ERR! stack at ChildProcess.onExit (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\lib\build.js:262:23)
gyp ERR! stack at emitTwo (events.js:126:13)
gyp ERR! stack at ChildProcess.emit (events.js:214:7)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
gyp ERR! System Windows_NT 10.0.17134
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\node_modules\\no de-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd C:\Users\Jerry\AppData\Local\Yarn\Data\global\node _modules\better-sqlite3
yarn global v1.10.1
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
[4/5] ⠄ better-sqlite3
[2/5] ⠄ puppeteer
[-/5] ⠄ waiting...
[-/5] ⠄ waiting...
error C:\Users\Jerry\AppData\Local\Yarn\Data\global\node _modules\better-sqlite3: Command failed.
Exit code: 1
Command: node-gyp rebuild
Arguments:
Directory: C:\Users\Jerry\AppData\Local\Yarn\Data\global\node _modules\better-sqlite3
Output:
C:\Users\Jerry\AppData\Local\Yarn\Data\global\node _modules\better-sqlite3>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modules\npm\bin\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild ) else (node "" rebuild )
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] win32 x64
gyp info spawn C:\Users\Jerry\.windows-build-tools\python27\python.EXE
gyp info spawn args [ 'C:\\Program Files\\nodejs\\node_modules\\npm\\node_modules\\no de-gyp\\gyp\\gyp_main.py',
gyp info spawn args 'binding.gyp',
gyp info spawn args '-f',
gyp info spawn args 'msvs',
gyp info spawn args '-G',
gyp info spawn args 'msvs_version=2015',
gyp info spawn args '-I',
gyp info spawn args 'C:\\Users\\Jerry\\AppData\\Local\\Yarn\\Data\\glo bal\\node_modules\\better-sqlite3\\build\\config.gypi',
gyp info spawn args '-I',
gyp info spawn args 'C:\\Program Files\\nodejs\\node_modules\\npm\\node_modules\\no de-gyp\\addon.gypi',
gyp info spawn args '-I',
gyp info spawn args 'C:\\Users\\Jerry\\.node-gyp\\8.12.0\\include\\node\\common.gypi',
gyp info spawn args '-Dlibrary=shared_library',
gyp info spawn args '-Dvisibility=default',
gyp info spawn args '-Dnode_root_dir=C:\\Users\\Jerry\\.node-gyp\\8.12.0',
gyp info spawn args '-Dnode_gyp_dir=C:\\Program Files\\nodejs\\node_modules\\npm\\node_modules\\no de-gyp',
gyp info spawn args '-Dnode_lib_file=C:\\Users\\Jerry\\.node-gyp\\8.12.0\\<(target_arch)\\node.lib',
gyp info spawn args '-Dmodule_root_dir=C:\\Users\\Jerry\\AppData\\Local\ \Yarn\\Data\\global\\node_modules\\better-sqlite3',
gyp info spawn args '-Dnode_engine=v8',
gyp info spawn args '--depth=.',
gyp info spawn args '--no-parallel',
gyp info spawn args '--generator-output',
gyp info spawn args 'C:\\Users\\Jerry\\AppData\\Local\\Yarn\\Data\\glo bal\\node_modules\\better-sqlite3\\build',
gyp info spawn args '-Goutput_dir=.' ]
gyp info spawn C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\15.0\Bin\MSBuild.ex e
gyp info spawn args [ 'build/binding.sln',
gyp info spawn args '/clp:Verbosity=minimal',
gyp info spawn args '/nologo',
gyp info spawn args '/p:Configuration=Release;Platform=x64' ]
Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\Common7\IDE\VC\VCTargets\Mi crosoft.Cpp.Platform.targets(67,5): error MSB8020: The build tools for v140 (Platform Toolset = 'v140') cannot be found. To build using the v140 build tools, please install v140 build tools. Alternatively, you may upgrade to the current Visual Studio tools by selecting the Project menu or right-click the solution, and then selecting "Retarget solution". [C:\Users\Jerry\AppData\Local\Yarn\Data\global\node _modules\better-sqlite3\build\deps\action_before_build.vcxproj]
gyp ERR! build error
gyp ERR! stack Error: `C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\15.0\Bin\MSBuild.ex e` failed with exit code: 1
gyp ERR! stack at ChildProcess.onExit (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\lib\build.js:262:23)
gyp ERR! stack at emitTwo (events.js:126:13)
gyp ERR! stack at ChildProcess.emit (events.js:214:7)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
gyp ERR! System Windows_NT 10.0.17134
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\node_modules\\no de-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd C:\Users\Jerry\AppData\Local\Yarn\Data\global\node _modules\better-sqlite3
but upon running, it gives a different error:
flightplan-search(1) does not exist, try --help
#147
Original Poster
Join Date: Jan 2010
Posts: 189
Hello! The demo is impressive and love what you did on visualization, great job!!
I want to try it out myself but could not get it to work on either windows10 or maxos mohave.
Windows 10:
node & yarn installed successfully
upon executing 'yarn global add flightplan-tool', it throws an error on 'better-sqlite3'
I want to try it out myself but could not get it to work on either windows10 or maxos mohave.
Windows 10:
node & yarn installed successfully
upon executing 'yarn global add flightplan-tool', it throws an error on 'better-sqlite3'
npm install --add-python-to-path --global --production windows-build-tools
I'm running Mojave myself, and not seeing it, but it seems related to the latest version of yarn. There'll be a fix for it, either tonight or tomorrow. If you open Terminal, run the below command and paste the output here, I can confirm if it's the same issue or not.
ls -l `which flightplan`
#148
Original Poster
Join Date: Jan 2010
Posts: 189
Sorry, forgot one more important step in the instructions You should delete your "data" and "db" directories, the database format is totally different in the new version. Flightplan's picking up the old database, and getting confused by it.
#149
Join Date: Feb 2013
Location: DTW
Programs: DL Diamond/AA Platinum, Hyatt/Hilton Diamond
Posts: 245
The error is saying it couldn't find VS build tools, did you maybe forget to install them? In the README, it's this step:
That error looks suspiciously similar to the one ferni filed an issue for: https://github.com/flightplan-tool/flightplan/issues/7
I'm running Mojave myself, and not seeing it, but it seems related to the latest version of yarn. There'll be a fix for it, either tonight or tomorrow. If you open Terminal, run the below command and paste the output here, I can confirm if it's the same issue or not.
That error looks suspiciously similar to the one ferni filed an issue for: https://github.com/flightplan-tool/flightplan/issues/7
I'm running Mojave myself, and not seeing it, but it seems related to the latest version of yarn. There'll be a fix for it, either tonight or tomorrow. If you open Terminal, run the below command and paste the output here, I can confirm if it's the same issue or not.
PS C:\WINDOWS\system32> npm install --add-python-to-path --global --production windows-build-tools
> [email protected] postinstall C:\Users\Jerry\AppData\Roaming\npm\node_modules\wi ndows-build-tools
> node ./dist/index.js
Downloading vs_BuildTools.exe
[> ] 0.0% (0 B/s)
Downloaded vs_BuildTools.exe. Saved to C:\Users\Jerry\.windows-build-tools\vs_BuildTools.exe.
Starting installation...
Launched installers, now waiting for them to finish.
This will likely take some time - please be patient!
Status from the installers:
---------- Visual Studio Build Tools ----------
Successfully installed Visual Studio Build Tools.
------------------- Python --------------------
Python 2.7.15 is already installed, not installing again.
Now configuring the Visual Studio Build Tools..
All done!
+ [email protected]
updated 2 packages in 9.691s
> [email protected] postinstall C:\Users\Jerry\AppData\Roaming\npm\node_modules\wi ndows-build-tools
> node ./dist/index.js
Downloading vs_BuildTools.exe
[> ] 0.0% (0 B/s)
Downloaded vs_BuildTools.exe. Saved to C:\Users\Jerry\.windows-build-tools\vs_BuildTools.exe.
Starting installation...
Launched installers, now waiting for them to finish.
This will likely take some time - please be patient!
Status from the installers:
---------- Visual Studio Build Tools ----------
Successfully installed Visual Studio Build Tools.
------------------- Python --------------------
Python 2.7.15 is already installed, not installing again.
Now configuring the Visual Studio Build Tools..
All done!
+ [email protected]
updated 2 packages in 9.691s
Mac: I am away from my macbook. Will post later today.
#150
Original Poster
Join Date: Jan 2010
Posts: 189
Two more things to check: did you run as "admin" when you installed in powershell the first time? Also did you restart after installing the build tools?