Details
-
Type:
Task
-
Status: Closed (View Workflow)
-
Priority:
Major
-
Resolution: Done
-
Affects Version/s: None
-
Fix Version/s: 9.0.1 Minor Release
-
Labels:
-
Story Points:2
-
Sprint:Fall 2017
Description
When developers build the master branch in Netbeans, multiple tests fail.
IGB still seems to run just fine, but tests failing is a danger sign.
We need to determine what's wrong, and fix it.
Attachments
Issue Links
Activity
Build with tests results:
Reactor Summary:
IGB Project ....................................... SUCCESS [0.167s]
Heatmap Editor .................................... SUCCESS [4.173s]
Affymetrix Common ................................. SUCCESS [14.553s]
external-sort ..................................... SUCCESS [2.943s]
cache-api ......................................... SUCCESS [0.423s]
SynonymLookup ..................................... SUCCESS [4.782s]
genometry ......................................... SUCCESS [12.158s]
IGB Preferences ................................... SUCCESS [3.407s]
igbSwingExt ....................................... SUCCESS [1.671s]
IGB Genoviz Extensions ............................ SUCCESS [0.448s]
IGB Services ...................................... SUCCESS [0.335s]
frameManager ...................................... SUCCESS [0.338s]
Window Service .................................... SUCCESS [0.447s]
igb-javafx-util ................................... SUCCESS [0.463s]
Menu API .......................................... SUCCESS [0.374s]
Context Menu Service .............................. SUCCESS [0.412s]
igb ............................................... SKIPPED
shared-lib-wrapper ................................ SUCCESS [10.610s]
Logging Config .................................... SUCCESS [0.334s]
WebLinks .......................................... SUCCESS [0.782s]
cache ............................................. SUCCESS [2.056s]
Quickload ......................................... FAILURE [7.534s]
Das ............................................... SUCCESS [6.389s]
Das 2 ............................................. SUCCESS [1.583s]
Apollo Library .................................... SUCCESS [0.382s]
Track Operations .................................. SKIPPED
Annotation Tab .................................... SKIPPED
BigBedHandler ..................................... SUCCESS [0.176s]
BigWigHandler ..................................... SUCCESS [0.335s]
Bookmark .......................................... SKIPPED
External View ..................................... SUCCESS [1.267s]
Graph Tab ......................................... SKIPPED
image-exporter .................................... SUCCESS [1.593s]
IGB Script ........................................ SKIPPED
KeyWord Search .................................... SUCCESS [0.303s]
NCBI Blast ........................................ SKIPPED
Property View ..................................... SUCCESS [0.394s]
Restriction Sites ................................. SUCCESS [0.448s]
searchModelIdOrProps .............................. SUCCESS [0.371s]
searchView ........................................ SKIPPED
sequenceViewer .................................... SKIPPED
survey ............................................ SUCCESS [0.301s]
thresholding ...................................... SKIPPED
update ............................................ SUCCESS [0.317s]
Plugin Manager .................................... SKIPPED
Paired End Data Preprocessor ...................... SUCCESS [0.218s]
Console Gui ....................................... SUCCESS [0.633s]
Session Manager ................................... SKIPPED
Keybinding-registry ............................... SUCCESS [0.796s]
org.lorainelab.igb.error.reporter.manager ......... SUCCESS [0.387s]
feedback .......................................... SKIPPED
narrowpeak ........................................ SUCCESS [0.374s]
bed ............................................... SUCCESS [3.579s]
bam ............................................... SUCCESS [3.351s]
main .............................................. SKIPPED
------------------------------------------------------------------------
BUILD FAILURE
------------------------------------------------------------------------
Total time: 56.996s (Wall Clock)
Finished at: Wed May 17 10:45:30 EDT 2017
Final Memory: 62M/629M
------------------------------------------------------------------------
Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19:test (default-test) on project quickload: There are test failures.
Quickload failure was because of this method in QuickloadDataProviderTest:
public void testAssemblyInfo()
{ Map<String, Integer> assemblyInfo = dataProvider.getAssemblyInfo(version); assertTrue(assemblyInfo.containsKey("chr5")); assertTrue(assemblyInfo.containsKey("chrM")); }In QuickloadDataProvider.java, the function AssemblyInfo returns keys with the syntax "Chr5". Changed method testAssemblyInfo to expect this, and that resolved the issue.
Test output after fixing Quickload:
Reactor Summary:
IGB Project ....................................... SUCCESS [0.182s]
Heatmap Editor .................................... SUCCESS [6.089s]
Affymetrix Common ................................. SUCCESS [14.541s]
external-sort ..................................... SUCCESS [3.575s]
cache-api ......................................... SUCCESS [0.536s]
SynonymLookup ..................................... SUCCESS [2.545s]
genometry ......................................... SUCCESS [19.648s]
IGB Preferences ................................... SUCCESS [2.466s]
igbSwingExt ....................................... SUCCESS [1.829s]
IGB Genoviz Extensions ............................ SUCCESS [0.935s]
IGB Services ...................................... SUCCESS [0.519s]
frameManager ...................................... SUCCESS [0.395s]
Window Service .................................... SUCCESS [0.752s]
igb-javafx-util ................................... SUCCESS [0.420s]
Menu API .......................................... SUCCESS [0.467s]
Context Menu Service .............................. SUCCESS [0.292s]
igb ............................................... SUCCESS [10.194s]
shared-lib-wrapper ................................ SUCCESS [11.337s]
Logging Config .................................... SUCCESS [0.245s]
WebLinks .......................................... SUCCESS [1.365s]
cache ............................................. SUCCESS [2.289s]
Quickload ......................................... SUCCESS [7.720s]
Das ............................................... SUCCESS [7.054s]
Das 2 ............................................. SUCCESS [2.077s]
Apollo Library .................................... SUCCESS [0.882s]
Track Operations .................................. SUCCESS [0.945s]
Annotation Tab .................................... SKIPPED
BigBedHandler ..................................... SUCCESS [0.241s]
BigWigHandler ..................................... SUCCESS [0.231s]
Bookmark .......................................... SUCCESS [1.925s]
External View ..................................... SUCCESS [2.048s]
Graph Tab ......................................... SKIPPED
image-exporter .................................... SUCCESS [1.921s]
IGB Script ........................................ SUCCESS [0.525s]
KeyWord Search .................................... SUCCESS [0.353s]
NCBI Blast ........................................ FAILURE [1.985s]
Property View ..................................... SUCCESS [0.624s]
Restriction Sites ................................. SUCCESS [0.571s]
searchModelIdOrProps .............................. SUCCESS [0.642s]
searchView ........................................ SUCCESS [1.140s]
sequenceViewer .................................... SUCCESS [0.598s]
survey ............................................ SUCCESS [0.657s]
thresholding ...................................... SKIPPED
update ............................................ SUCCESS [0.504s]
Plugin Manager .................................... SKIPPED
Paired End Data Preprocessor ...................... SUCCESS [0.318s]
Console Gui ....................................... SUCCESS [0.588s]
Session Manager ................................... SKIPPED
Keybinding-registry ............................... SUCCESS [1.017s]
org.lorainelab.igb.error.reporter.manager ......... SUCCESS [0.478s]
feedback .......................................... SKIPPED
narrowpeak ........................................ SUCCESS [0.536s]
bed ............................................... SUCCESS [3.589s]
bam ............................................... SUCCESS [3.145s]
main .............................................. SKIPPED
------------------------------------------------------------------------
BUILD FAILURE
------------------------------------------------------------------------
Total time: 1:08.408s (Wall Clock)
Finished at: Wed May 17 14:32:30 EDT 2017
Final Memory: 84M/604M
NCBI Blast ........................................ FAILURE [1.985s]
Error log:
-------------------------------------------------------------------------------
Test set: apollo.analysis.RemoteBlastNCBITest
-------------------------------------------------------------------------------
Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.56 sec <<< FAILURE! - in apollo.analysis.RemoteBlastNCBITest
residueLengthLessThanLength(apollo.analysis.RemoteBlastNCBITest) Time elapsed: 0.484 sec <<< ERROR!
java.io.IOException: Server returned HTTP response code: 403 for URL: http://blast.ncbi.nlm.nih.gov/Blast.cgi?
at apollo.analysis.RemoteBlastNCBITest.checkRunAnalysis(RemoteBlastNCBITest.java:29)
at apollo.analysis.RemoteBlastNCBITest.residueLengthLessThanLength(RemoteBlastNCBITest.java:53)
residueLengthGreaterThanLength(apollo.analysis.RemoteBlastNCBITest) Time elapsed: 0.076 sec <<< ERROR!
java.io.IOException: Server returned HTTP response code: 403 for URL: http://blast.ncbi.nlm.nih.gov/Blast.cgi?
at apollo.analysis.RemoteBlastNCBITest.checkRunAnalysis(RemoteBlastNCBITest.java:29)
at apollo.analysis.RemoteBlastNCBITest.residueLengthGreaterThanLength(RemoteBlastNCBITest.java:60)
When attempting to use this feature (On arabidopsis thaliana, right clicked and chose option 'BLASTx nr protein database' ), I get the error:
"Server returned HTTP response code: 403 for URL: http://blast.ncbi.nlm.nih.gov/Blast.cgi?" This is a valid URL when copied into my browser (Firefox and Chrome).
Created issue IGBF-1154 for this issue since the fix may involve editing non-test files in IGB.
Resolved issue IGBF-1154. All tests now passing.
Added to 'needs 1st level review' to confirm that all tests pass.
Only code changed in this issue specifically is to fix the quickview test.
$ mvn clean install
...
main:
[jar] Building jar: /Users/lorainelab/Documents/Repos/integrated-genome-browser/igb_exe.jar
[INFO] Executed tasks
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] IGB Project ....................................... SUCCESS [ 0.282 s]
[INFO] Heatmap Editor .................................... SUCCESS [ 5.950 s]
[INFO] Affymetrix Common ................................. SUCCESS [ 7.436 s]
[INFO] external-sort ..................................... SUCCESS [ 2.396 s]
[INFO] cache-api ......................................... SUCCESS [ 0.539 s]
[INFO] SynonymLookup ..................................... SUCCESS [ 2.043 s]
[INFO] genometry ......................................... SUCCESS [ 17.252 s]
[INFO] IGB Preferences ................................... SUCCESS [ 1.877 s]
[INFO] igbSwingExt ....................................... SUCCESS [ 1.465 s]
[INFO] IGB Genoviz Extensions ............................ SUCCESS [ 0.898 s]
[INFO] IGB Services ...................................... SUCCESS [ 0.634 s]
[INFO] frameManager ...................................... SUCCESS [ 0.592 s]
[INFO] Window Service .................................... SUCCESS [ 0.883 s]
[INFO] igb-javafx-util ................................... SUCCESS [ 0.348 s]
[INFO] Menu API .......................................... SUCCESS [ 0.592 s]
[INFO] Context Menu Service .............................. SUCCESS [ 0.765 s]
[INFO] igb ............................................... SUCCESS [ 10.221 s]
[INFO] shared-lib-wrapper ................................ SUCCESS [ 6.099 s]
[INFO] Logging Config .................................... SUCCESS [ 0.408 s]
[INFO] WebLinks .......................................... SUCCESS [ 1.512 s]
[INFO] cache ............................................. SUCCESS [ 1.917 s]
[INFO] Quickload ......................................... SUCCESS [ 4.006 s]
[INFO] Das ............................................... SUCCESS [ 4.243 s]
[INFO] Das 2 ............................................. SUCCESS [ 1.469 s]
[INFO] Apollo Library .................................... SUCCESS [ 0.575 s]
[INFO] Track Operations .................................. SUCCESS [ 0.743 s]
[INFO] Annotation Tab .................................... SUCCESS [ 0.698 s]
[INFO] BigBedHandler ..................................... SUCCESS [ 0.392 s]
[INFO] BigWigHandler ..................................... SUCCESS [ 0.332 s]
[INFO] Bookmark .......................................... SUCCESS [ 1.054 s]
[INFO] External View ..................................... SUCCESS [ 1.552 s]
[INFO] Graph Tab ......................................... SUCCESS [ 0.565 s]
[INFO] image-exporter .................................... SUCCESS [ 1.615 s]
[INFO] IGB Script ........................................ SUCCESS [ 0.348 s]
[INFO] KeyWord Search .................................... SUCCESS [ 0.256 s]
[INFO] NCBI Blast ........................................ SUCCESS [ 6.687 s]
[INFO] Property View ..................................... SUCCESS [ 0.472 s]
[INFO] Restriction Sites ................................. SUCCESS [ 0.565 s]
[INFO] searchModelIdOrProps .............................. SUCCESS [ 0.534 s]
[INFO] searchView ........................................ SUCCESS [ 0.813 s]
[INFO] sequenceViewer .................................... SUCCESS [ 0.556 s]
[INFO] survey ............................................ SUCCESS [ 0.342 s]
[INFO] thresholding ...................................... SUCCESS [ 0.592 s]
[INFO] update ............................................ SUCCESS [ 0.276 s]
[INFO] Plugin Manager .................................... SUCCESS [ 2.451 s]
[INFO] Paired End Data Preprocessor ...................... SUCCESS [ 0.259 s]
[INFO] Console Gui ....................................... SUCCESS [ 0.499 s]
[INFO] Session Manager ................................... SUCCESS [ 0.475 s]
[INFO] Keybinding-registry ............................... SUCCESS [ 0.730 s]
[INFO] org.lorainelab.igb.error.reporter.manager ......... SUCCESS [ 0.278 s]
[INFO] feedback .......................................... SUCCESS [ 0.355 s]
[INFO] narrowpeak ........................................ SUCCESS [ 0.307 s]
[INFO] bed ............................................... SUCCESS [ 2.331 s]
[INFO] bam ............................................... SUCCESS [ 2.010 s]
[INFO] main .............................................. SUCCESS [ 7.164 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:51 min
[INFO] Finished at: 2017-05-24T15:09:51-05:00
[INFO] Final Memory: 148M/1550M
[INFO] ------------------------------------------------------------------------
$
Great! the Quickload test looks good--they all do!
I am still curious... why was this broken?
I know IGB supports synonyms for chromosome names and I think it is supposed to support "Chr1" and "chr1" for the A. thaliana genome. Might this test have been designed to test that the synonym system is working? And the test failing is indicating that the synonym system is NOT working...? I doesn't say that, but this file is pretty void of comments, so we have to make assumptions about what each test is designed to test for.
In an older version (8.2.3) the chromosome names that display in the A. thaliana genome are lower cased, "chr1".
This is odd, given that IGB doesn't store data. So both versions that I have running next to each other (v8.2 and v9.0) should be looking at the same data, because they are looking at the same remote quickload site. But in v8.2 shows "chr1" and v9.0 shows "Chr1".
The "Chr1" is a more accurate reflection of what is in the genome file. So how does v8.2 have "chr1" ?
The user's guide talks about making a chromosomes.txt file with synonyms for chromosome names.
https://wiki.transvar.org/display/igbman/Personal+Synonyms
I think IGB has a built-in chromosomes.txt file.
I think there must have been a change in that file or the code that uses it.
We should test the synonyms functionality.
The current solution is good. It checks that the expected chromosome names are present (based on the names they have in the sequence file). And making the test pass allows all of the subsequent tests to run.
First level review #1:
The code looks good, simple, logical.
The outcome (tests passing) is good.
I'm kicking this back to To Do for minor changes before going to the second "1st level review"
-There are no comments around the change. I think we are currently operating on a comment-trigger-happy MO.
-I've requested changes in the git history (details in email).
Jenn, consider those changes, talk with Ann if needed. Then move to "1st level review" to get the second review.
Re-worked this branch to exclude a stashed commit. Also added a comment above the changes made to QuickloadDataProviderTest to mark the changes made per this issue.
This branch is now 2 commits ahead of master. Moving to 'needs review' for the second review.
Done with Second level review.
Code changes are precise and good. I am now having a successful build.
------------------------------------------------------------------------------------------------------------------
Reactor Summary:
IGB Project ....................................... SUCCESS [0.252s]
Heatmap Editor .................................... SUCCESS [8.739s]
Affymetrix Common ................................. SUCCESS [14.532s]
external-sort ..................................... SUCCESS [9.999s]
cache-api ......................................... SUCCESS [7.937s]
SynonymLookup ..................................... SUCCESS [6.259s]
genometry ......................................... SUCCESS [13.987s]
IGB Preferences ................................... SUCCESS [5.502s]
igbSwingExt ....................................... SUCCESS [2.751s]
IGB Genoviz Extensions ............................ SUCCESS [3.075s]
IGB Services ...................................... SUCCESS [1.827s]
frameManager ...................................... SUCCESS [1.032s]
Window Service .................................... SUCCESS [1.823s]
igb-javafx-util ................................... SUCCESS [0.864s]
Menu API .......................................... SUCCESS [2.804s]
Context Menu Service .............................. SUCCESS [0.818s]
igb ............................................... SUCCESS [23.155s]
shared-lib-wrapper ................................ SUCCESS [14.759s]
Logging Config .................................... SUCCESS [0.764s]
WebLinks .......................................... SUCCESS [3.376s]
cache ............................................. SUCCESS [4.635s]
Quickload ......................................... SUCCESS [7.306s]
Das ............................................... SUCCESS [6.862s]
Das 2 ............................................. SUCCESS [3.751s]
Apollo Library .................................... SUCCESS [1.813s]
Track Operations .................................. SUCCESS [1.908s]
Annotation Tab .................................... SUCCESS [1.175s]
BigBedHandler ..................................... SUCCESS [0.748s]
BigWigHandler ..................................... SUCCESS [0.843s]
Bookmark .......................................... SUCCESS [2.162s]
External View ..................................... SUCCESS [4.000s]
Graph Tab ......................................... SUCCESS [1.495s]
image-exporter .................................... SUCCESS [3.748s]
IGB Script ........................................ SUCCESS [1.030s]
KeyWord Search .................................... SUCCESS [1.114s]
NCBI Blast ........................................ SUCCESS [9.439s]
Property View ..................................... SUCCESS [1.810s]
Restriction Sites ................................. SUCCESS [1.228s]
searchModelIdOrProps .............................. SUCCESS [0.784s]
searchView ........................................ SUCCESS [1.979s]
sequenceViewer .................................... SUCCESS [0.926s]
survey ............................................ SUCCESS [1.163s]
thresholding ...................................... SUCCESS [1.375s]
update ............................................ SUCCESS [0.941s]
Plugin Manager .................................... SUCCESS [3.749s]
Paired End Data Preprocessor ...................... SUCCESS [0.621s]
Console Gui ....................................... SUCCESS [1.530s]
Session Manager ................................... SUCCESS [0.667s]
Keybinding-registry ............................... SUCCESS [2.119s]
org.lorainelab.igb.error.reporter.manager ......... SUCCESS [2.009s]
feedback .......................................... SUCCESS [0.795s]
narrowpeak ........................................ SUCCESS [1.692s]
bed ............................................... SUCCESS [6.489s]
bam ............................................... SUCCESS [5.026s]
main .............................................. SUCCESS [13.935s]
------------------------------------------------------------------------
BUILD SUCCESS
------------------------------------------------------------------------
Total time: 1:34.606s (Wall Clock)
Finished at: Wed May 31 12:39:08 EDT 2017
Final Memory: 54M/796M
------------------------------------------------------------------------
Issue is ready for a pull request.
I tested this issue on the master branch (since the description says that the problem relates to developers running from the master branch) and I tested this issue on the IGBF-1142 branch of Jenny's fork. I realize that the fix for this issue must not be incorporated into the master branch yet, because I am experiencing different behavior on the branches when building from Netbeans.
On the IGBF-1142 branch, I am able to successfully build IGB from Netbeans without skipping tests. The build is successful on my Windows and Mac. For this reason, I believe this JIRA story to be resolved.
On the master branch I am running into something odd: The build is successful on my Windows machine but fails on my Mac. I am trying to look into why this is but do not have any answers yet. I have reached out to Jenny to see why this may be, but I don't think any fix for this issue has been merged into the master branch yet, so I still believe this JIRA story to be resolved although I do think the behavior is odd.
Since the build is successful on the IGBF-1142 branch on both my Windows and Mac, I am marking this issue as resolved. If I gain new information that the fix should have been merged into the master branch already, I will re-open the issue and continue looking into it.
Link to this branch on Bitbucket:
https://bitbucket.org/jdaly8/integrated-genome-browser/branch/IGBF-1142