Scala学习10之在eclipse下使用maven对spark和adam中的scalatest进行测试
靳富
Spark和Adam中有用scalatest来进行test
1.使用maven来进行编译请见【1】，
（1）常见的
mvn test
mvn test -DskipTests
都没问题
但是不知道怎么之测试一个suite？？
（2）指定的： 参考【3】中的，【1】中有更多说明
eclipse中：
run as ->build->
-DwildcardSuites=org.bdgenomics.adam.converters.AlignmentRecordConverterSuite test即在命令行为：mvn DwildcardSuites=org.bdgenomics.adam.converters.AlignmentRecordConverterSuite test
结果：
[WARNING]
[WARNING] Some problems were encountered while building the effective settings
[WARNING] Unrecognised tag: 'profile' (position: START_TAG seen ... variables for plugins in the POM.\n |\n |-->\n \t\t\t<profile>... @184:16) @ D:\1win7\java\apache-maven-3.3.9-bin\apache-maven-3.3.9\conf\settings.xml, line 184, column 16
[WARNING]
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building ADAM_2.10: Core 0.19.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:1.0:enforce (enforce-versions) @ adam-core_2.10 ---
[INFO]
[INFO] --- maven-enforcer-plugin:1.0:enforce (enforce-maven) @ adam-core_2.10 ---
[INFO]
[INFO] --- build-helper-maven-plugin:1.9.1:add-source (add-source) @ adam-core_2.10 ---
[INFO] Source directory: D:\all\eclipse432\adam-2.10-0.19-git-bin\adam-2.10-0.19-git\adam-core\src\main\scala added.
[INFO]
[INFO] --- scalariform-maven-plugin:0.1.4:format (default-cli) @ adam-core_2.10 ---
[INFO] Modified 0 of 159 .scala files
[INFO]
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ adam-core_2.10 ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
[INFO] --- scala-maven-plugin:3.2.2:compile (scala-compile-first) @ adam-core_2.10 ---
[WARNING] Expected all dependencies to require Scala version: 2.10.4
[WARNING] org.bdgenomics.utils:utils-misc_2.10:0.2.4 requires scala version: 2.10.4
[WARNING] org.bdgenomics.utils:utils-misc_2.10:0.2.4 requires scala version: 2.10.4
[WARNING] org.bdgenomics.utils:utils-metrics_2.10:0.2.4 requires scala version: 2.10.4
[WARNING] org.bdgenomics.utils:utils-io_2.10:0.2.4 requires scala version: 2.10.4
[WARNING] org.bdgenomics.utils:utils-cli_2.10:0.2.4 requires scala version: 2.10.4
[WARNING] org.scoverage:scalac-scoverage-plugin_2.10:1.1.1 requires scala version: 2.10.4
[WARNING] org.bdgenomics.adam:adam-core_2.10:0.19.0 requires scala version: 2.10.4
[WARNING] com.twitter:chill_2.10:0.5.0 requires scala version: 2.10.4
[WARNING] com.typesafe.akka:akka-remote_2.10:2.3.11 requires scala version: 2.10.4
[WARNING] com.typesafe.akka:akka-actor_2.10:2.3.11 requires scala version: 2.10.4
[WARNING] com.typesafe.akka:akka-slf4j_2.10:2.3.11 requires scala version: 2.10.4
[WARNING] org.apache.spark:spark-core_2.10:1.5.2 requires scala version: 2.10.4
[WARNING] org.json4s:json4s-jackson_2.10:3.2.10 requires scala version: 2.10.4
[WARNING] org.json4s:json4s-core_2.10:3.2.10 requires scala version: 2.10.4
[WARNING] org.json4s:json4s-ast_2.10:3.2.10 requires scala version: 2.10.4
[WARNING] org.json4s:json4s-core_2.10:3.2.10 requires scala version: 2.10.0
[WARNING] Multiple versions of scala libraries detected!
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-compiler-plugin:3.3:compile (default-compile) @ adam-core_2.10 ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- build-helper-maven-plugin:1.9.1:add-test-source (add-test-source) @ adam-core_2.10 ---
[INFO] Test Source directory: D:\all\eclipse432\adam-2.10-0.19-git-bin\adam-2.10-0.19-git\adam-core\src\test\scala added.
[INFO]
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ adam-core_2.10 ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 65 resources
[INFO]
[INFO] --- scala-maven-plugin:3.2.2:testCompile (scala-test-compile-first) @ adam-core_2.10 ---
[WARNING] Expected all dependencies to require Scala version: 2.10.4
[WARNING] org.bdgenomics.utils:utils-misc_2.10:0.2.4 requires scala version: 2.10.4
[WARNING] org.bdgenomics.utils:utils-misc_2.10:0.2.4 requires scala version: 2.10.4
[WARNING] org.bdgenomics.utils:utils-metrics_2.10:0.2.4 requires scala version: 2.10.4
[WARNING] org.bdgenomics.utils:utils-io_2.10:0.2.4 requires scala version: 2.10.4
[WARNING] org.bdgenomics.utils:utils-cli_2.10:0.2.4 requires scala version: 2.10.4
[WARNING] org.scoverage:scalac-scoverage-plugin_2.10:1.1.1 requires scala version: 2.10.4
[WARNING] org.bdgenomics.adam:adam-core_2.10:0.19.0 requires scala version: 2.10.4
[WARNING] com.twitter:chill_2.10:0.5.0 requires scala version: 2.10.4
[WARNING] com.typesafe.akka:akka-remote_2.10:2.3.11 requires scala version: 2.10.4
[WARNING] com.typesafe.akka:akka-actor_2.10:2.3.11 requires scala version: 2.10.4
[WARNING] com.typesafe.akka:akka-slf4j_2.10:2.3.11 requires scala version: 2.10.4
[WARNING] org.apache.spark:spark-core_2.10:1.5.2 requires scala version: 2.10.4
[WARNING] org.json4s:json4s-jackson_2.10:3.2.10 requires scala version: 2.10.4
[WARNING] org.json4s:json4s-core_2.10:3.2.10 requires scala version: 2.10.4
[WARNING] org.json4s:json4s-ast_2.10:3.2.10 requires scala version: 2.10.4
[WARNING] org.json4s:json4s-core_2.10:3.2.10 requires scala version: 2.10.0
[WARNING] Multiple versions of scala libraries detected!
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-compiler-plugin:3.3:testCompile (default-testCompile) @ adam-core_2.10 ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ adam-core_2.10 ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- scalatest-maven-plugin:1.0:test (test) @ adam-core_2.10 ---
[36mDiscovery starting.[0m
[36mDiscovery completed in 50 seconds, 526 milliseconds.[0m
[36mRun starting. Expected test count is: 7[0m
[32mAlignmentRecordConverterSuite:[0m
[32m- testing the fields in a converted ADAM Read[0m
[32m- converting a read with null quality is OK[0m
[32m- convert a read to fastq[0m
[32m- reverse complement reads when converting to fastq[0m
[32m- converting to fastq with unmapped reads[0m
[32m- converting a fragment with no alignments should yield unaligned reads[0m
[32m- converting a fragment with alignments should restore the alignments[0m
[36mRun completed in 54 seconds, 557 milliseconds.[0m
[36mTotal number of tests run: 7[0m
[36mSuites: completed 2, aborted 0[0m
[36mTests: succeeded 7, failed 0, canceled 0, ignored 0, pending 0[0m
[32mAll tests passed.[0m
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:44 min
[INFO] Finished at: 2016-05-13T19:31:57+08:00
[INFO] Final Memory: 24M/207M
[INFO] ------------------------------------------------------------------------
附录测试源文件：
/**
* Licensed to Big Data Genomics (BDG) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The BDG licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bdgenomics.adam.converters
import htsjdk.samtools.{ SamReaderFactory, SAMRecord }
import java.io.File
import org.bdgenomics.adam.models.{
RecordGroupDictionary,
RecordGroup,
SAMFileHeaderWritable,
SequenceDictionary,
SequenceRecord
}
import org.bdgenomics.formats.avro.{
AlignmentRecord,
Contig,
Fragment
}
import org.scalatest.FunSuite
import scala.collection.JavaConversions._
class AlignmentRecordConverterSuite extends FunSuite {
// allocate converters
val adamRecordConverter = new AlignmentRecordConverter
def makeRead(start: Long, cigar: String, mdtag: String, length: Int, id: Int = 0, nullQuality: Boolean = false): AlignmentRecord = {
val sequence: String = "A" * length
val builder = AlignmentRecord.newBuilder()
.setReadName("read" + id.toString)
.setStart(start)
.setReadMapped(true)
.setCigar(cigar)
.setSequence(sequence)
.setReadNegativeStrand(false)
.setMapq(60)
.setMismatchingPositions(mdtag)
.setOldPosition(12)
.setOldCigar("2^AAA3")
if (!nullQuality) {
builder.setQual(sequence) // no typo, we just don't care
}
builder.build()
}
test("testing the fields in a converted ADAM Read") {
val adamRead = makeRead(3L, "2M3D2M", "2^AAA2", 4)
// add reference details
adamRead.setRecordGroupName("record_group")
adamRead.setRecordGroupSample("sample")
adamRead.setContig(Contig.newBuilder()
.setContigName("referencetest")
.build())
adamRead.setMateContig(Contig.newBuilder()
.setContigName("matereferencetest")
.setContigLength(6L)
.setReferenceURL("test://chrom1")
.build())
adamRead.setMateAlignmentStart(6L)
// make sequence dictionary
val seqRecForDict = SequenceRecord("referencetest", 5, "test://chrom1")
val dict = SequenceDictionary(seqRecForDict)
//make read group dictionary
val readGroup = new RecordGroup(adamRead.getRecordGroupSample(), adamRead.getRecordGroupName())
val readGroups = new RecordGroupDictionary(Seq(readGroup))
// convert read
val toSAM = adamRecordConverter.convert(adamRead,
SAMFileHeaderWritable(adamRecordConverter.createSAMHeader(dict,
readGroups)),
readGroups)
// validate conversion
val sequence = "A" * 4
assert(toSAM.getReadName === ("read" + 0.toString))
assert(toSAM.getAlignmentStart === 4)
assert(toSAM.getReadUnmappedFlag === false)
assert(toSAM.getCigarString === "2M3D2M")
assert(toSAM.getReadString === sequence)
assert(toSAM.getReadNegativeStrandFlag === false)
assert(toSAM.getMappingQuality === 60)
assert(toSAM.getBaseQualityString === sequence)
assert(toSAM.getAttribute("MD") === "2^AAA2")
assert(toSAM.getIntegerAttribute("OP") === 13)
assert(toSAM.getStringAttribute("OC") === "2^AAA3")
//make sure that we didn't set the SM attribute.
//issue #452 https://github.com/bigdatagenomics/adam/issues/452
assert(toSAM.getAttribute("SM") === null)
assert(toSAM.getHeader().getReadGroup("record_group").getSample() === "sample")
}
test("converting a read with null quality is OK") {
val adamRead = makeRead(3L, "2M3D2M", "2^AAA2", 4, nullQuality = true)
// add reference details
adamRead.setRecordGroupName("record_group")
adamRead.setRecordGroupSample("sample")
adamRead.setContig(Contig.newBuilder()
.setContigName("referencetest")
.build())
adamRead.setMateContig(Contig.newBuilder()
.setContigName("matereferencetest")
.setContigLength(6L)
.setReferenceURL("test://chrom1")
.build())
adamRead.setMateAlignmentStart(6L)
// make sequence dictionary
val seqRecForDict = SequenceRecord("referencetest", 5, "test://chrom1")
val dict = SequenceDictionary(seqRecForDict)
//make read group dictionary
val readGroup = new RecordGroup(adamRead.getRecordGroupSample(), adamRead.getRecordGroupName())
val readGroups = new RecordGroupDictionary(Seq(readGroup))
// convert read
val toSAM = adamRecordConverter.convert(adamRead,
SAMFileHeaderWritable(adamRecordConverter.createSAMHeader(dict,
readGroups)),
readGroups)
// validate conversion
val sequence = "A" * 4
assert(toSAM.getReadName === ("read" + 0.toString))
assert(toSAM.getAlignmentStart === 4)
assert(toSAM.getReadUnmappedFlag === false)
assert(toSAM.getCigarString === "2M3D2M")
assert(toSAM.getReadString === sequence)
assert(toSAM.getReadNegativeStrandFlag === false)
assert(toSAM.getMappingQuality === 60)
assert(toSAM.getBaseQualityString === "*")
assert(toSAM.getAttribute("MD") === "2^AAA2")
assert(toSAM.getIntegerAttribute("OP") === 13)
assert(toSAM.getStringAttribute("OC") === "2^AAA3")
//make sure that we didn't set the SM attribute.
//issue #452 https://github.com/bigdatagenomics/adam/issues/452
assert(toSAM.getAttribute("SM") === null)
assert(toSAM.getHeader().getReadGroup("record_group").getSample() === "sample")
}
test("convert a read to fastq") {
val adamRead = AlignmentRecord.newBuilder()
.setSequence("ACACCAACATG")
.setQual(".+**.+;:**.")
.setReadName("thebestread")
.build()
val fastq = adamRecordConverter.convertToFastq(adamRead)
.toString
.split('\n')
assert(fastq(0) === "@thebestread")
assert(fastq(1) === "ACACCAACATG")
assert(fastq(2) === "+")
assert(fastq(3) === ".+**.+;:**.")
}
def getSAMRecordFromReadName(readName: String): (AlignmentRecord, AlignmentRecord) = {
val samToADAMConverter = new SAMRecordConverter
val SAMTestFile = new File(getClass.getClassLoader.getResource("bqsr1.sam").getFile)
val newSAMReader = SamReaderFactory.makeDefault().open(SAMTestFile)
// Obtain SAMRecord
val newSAMRecord = newSAMReader.iterator().dropWhile(r => r.getReadName != readName)
val newSequenceRecord = SequenceRecord("22", 51304566)
val newSequenceDictionary = SequenceDictionary(newSequenceRecord)
val firstRecord = samToADAMConverter.convert(newSAMRecord.next(), newSequenceDictionary, new RecordGroupDictionary(Seq()))
val secondRecord = samToADAMConverter.convert(newSAMRecord.next(), newSequenceDictionary, new RecordGroupDictionary(Seq()))
(firstRecord, secondRecord)
}
test("reverse complement reads when converting to fastq") {
// SRR062634.10022079 83 22 16082719 0 5S95M = 16082635 -179
// AAGTAGCTGGGACTACACGCACGCACCACCATGCCTGGCTAATTTTTGTATTTTTAGTAGAGATGAGGTTTCACCATATTGGCCAGGCTGGTTTTGAATT
// #####EB5BB<840&:2?>A?-AC8=,5@AABCB?CEDBDC@6BB,CA0CB,B-DEDEDEDEA:D?DE5EBEC?E?5?D:AEEEDEDDEEE=BEEBDD-?
// RG:Z:SRR062634 XC:i:95 XT:A:R NM:i:2 SM:i:0 AM:i:0 X0:i:3 X1:i:0 XM:i:2 XO:i:0 XG:i:0 MD:Z:15G0T78
// XA:Z:GL000244.1,+31092,100M,2;14,+19760216,100M,2;
val (firstRecord, secondRecord) = getSAMRecordFromReadName("SRR062634.10022079")
assert(firstRecord.getReadInFragment === 1)
assert(secondRecord.getReadInFragment === 0)
val firstRecordFastq = adamRecordConverter.convertToFastq(firstRecord, maybeAddSuffix = true)
.toString
.split('\n')
assert(firstRecordFastq(0) === "@SRR062634.10022079/2")
assert(firstRecordFastq(1) === "CTGGAGTGCAGTGGCATGATTTCAGCTCACTGTCGTCTCTGCCTCCCTGACTCAAGTGATTCTCCTGCCTCAGCCTCCCACGTCGCTCGGACTCCACGCC")
assert(firstRecordFastq(2) === "+")
assert(firstRecordFastq(3) === "A:=D5D5E?D?DDD:.@@@@=?EE=DADDB@D=DD??ED=:CCCC?D:E=EEB=-C>C=@=EEEEB5EC-?A>=C-C?DC+34+4A>-?5:=/-A=@>>:")
val secondRecordFastq = adamRecordConverter.convertToFastq(secondRecord, maybeAddSuffix = true)
.toString
.split('\n')
assert(secondRecordFastq(0) === "@SRR062634.10022079/1")
assert(secondRecordFastq(1) === "AATTCAAAACCAGCCTGGCCAATATGGTGAAACCTCATCTCTACTAAAAATACAAAAATTAGCCAGGCATGGTGGTGCGTGCGTGTAGTCCCAGCTACTT")
assert(secondRecordFastq(2) === "+")
assert(secondRecordFastq(3) === "?-DDBEEB=EEEDDEDEEEA:D?5?E?CEBE5ED?D:AEDEDEDED-B,BC0AC,BB6@CDBDEC?BCBAA@5,=8CA-?A>?2:&048<BB5BE#####")
}
test("converting to fastq with unmapped reads") {
//SRR062634.10448889 117 22 16079761 0 * = 16079761 0
// TTTCTTTCTTTTATATATATATACACACACACACACACACACACACATATATGTATATATACACGTATATGTATGTATATATGTATATATACACGTATAT
// @DF>C;FDC=EGEGGEFDGEFDD?DFDEEGFGFGGGDGGGGGGGEGGGGFGGGFGGGGGGFGGFGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
// RG:Z:SRR062634
val (secondRecord, firstRecord) = getSAMRecordFromReadName("SRR062634.10448889")
assert(firstRecord.getReadInFragment === 1)
assert(secondRecord.getReadInFragment === 0)
val firstRecordFastq = adamRecordConverter.convertToFastq(firstRecord, maybeAddSuffix = true)
.toString
.split('\n')
assert(firstRecord.getReadMapped)
assert(firstRecord.getReadNegativeStrand)
assert(firstRecordFastq(0) === "@SRR062634.10448889/2")
assert(firstRecordFastq(1) === "ACCTGTCTCAGCCTCCCAAAGTGCTGCGATTACAGTCATGAGCCACCGCACTTGGCTGGGTTTTCGTTTTCTTTCTTTTATATATATATACACACACACA")
assert(firstRecordFastq(2) === "+")
assert(firstRecordFastq(3) === "GGGGGGGGGGGGGGGGGGGGGEGGGGGGGGGGGGGGGGGGGGGGGGGFGEGEEDGGFDF?AEEEBDADEEDEEE;DFC@'B:B=B=B=BADCBCBCA=DA")
val secondRecordFastq = adamRecordConverter.convertToFastq(secondRecord, maybeAddSuffix = true)
.toString
.split('\n')
assert(!secondRecord.getReadMapped)
assert(secondRecord.getReadNegativeStrand)
assert(secondRecordFastq(0) === "@SRR062634.10448889/1")
assert(secondRecordFastq(1) === secondRecord.getSequence)
assert(secondRecordFastq(2) === "+")
assert(secondRecordFastq(3) === secondRecord.getQual)
}
test("converting a fragment with no alignments should yield unaligned reads") {
val alignments = List(
AlignmentRecord.newBuilder()
.setSequence("ACCCACAGTA")
.setQual("**********")
.setReadInFragment(0)
.setReadName("testRead")
.setReadPaired(true)
.build(),
AlignmentRecord.newBuilder()
.setSequence("GGGAAACCCTTT")
.setQual(";;;;;;......")
.setReadName("testRead")
.setReadInFragment(1)
.setReadPaired(true)
.build())
val fragment = Fragment.newBuilder()
.setReadName("testRead")
.setAlignments(seqAsJavaList(alignments))
.build()
val reads = adamRecordConverter.convertFragment(fragment)
assert(reads.size === 2)
val read1 = reads.find(_.getReadInFragment == 0)
assert(read1.isDefined)
assert(read1.get.getSequence === "ACCCACAGTA")
assert(read1.get.getQual() === "**********")
assert(read1.get.getReadName === "testRead")
assert(!read1.get.getReadMapped)
assert(read1.get.getReadPaired)
val read2 = reads.find(_.getReadInFragment == 1)
assert(read2.isDefined)
assert(read2.get.getSequence === "GGGAAACCCTTT")
assert(read2.get.getQual() === ";;;;;;......")
assert(read2.get.getReadName === "testRead")
assert(!read2.get.getReadMapped)
assert(read2.get.getReadPaired)
}
test("converting a fragment with alignments should restore the alignments") {
val alignments = List(AlignmentRecord.newBuilder()
.setReadMapped(true)
.setContig(Contig.newBuilder()
.setContigName("1")
.build())
.setStart(10L)
.setEnd(20L)
.setReadName("testRead")
.setCigar("10M")
.setReadNegativeStrand(true)
.setSequence("TACTGTGGGT")
.setQual("?????*****")
.build())
val fragment = Fragment.newBuilder()
.setReadName("testRead")
.setAlignments(seqAsJavaList(alignments))
.build()
val reads = adamRecordConverter.convertFragment(fragment)
assert(reads.size === 1)
val read = reads.head
assert(read.getReadName === "testRead")
assert(read.getReadInFragment === 0)
assert(read.getReadMapped)
assert(read.getReadNegativeStrand)
assert(read.getStart === 10L)
assert(read.getEnd === 20L)
assert(read.getCigar === "10M")
assert(read.getSequence === "TACTGTGGGT")
assert(read.getQual === "?????*****")
assert(read.getContig.getContigName === "1")
}
}
参考：
【1】 http://www.scalatest.org/user_guide/using_the_scalatest_maven_plugin
【2】http://www.scalatest.org/user_guide/using_scalatest_with_eclipse
【3】https://github.com/scalatest/scalatest-eclipse-plugin
【4】http://blog.csdn.net/u011414200/article/details/49333757
类似资料：