当前位置: 首页 > 工具软件 > ScalaTest > 使用案例 >

Scala学习10之在eclipse下使用maven对spark和adam中的scalatest进行测试

靳富
2023-12-01

Spark和Adam中有用scalatest来进行test

1.使用maven来进行编译请见【1】,
(1)常见的
mvn test
mvn test -DskipTests
都没问题
但是不知道怎么之测试一个suite??
(2)指定的: 参考【3】中的,【1】中有更多说明
eclipse中:
run as ->build->

-DwildcardSuites=org.bdgenomics.adam.converters.AlignmentRecordConverterSuite test

即在命令行为:mvn DwildcardSuites=org.bdgenomics.adam.converters.AlignmentRecordConverterSuite test
结果:

[WARNING] 
[WARNING] Some problems were encountered while building the effective settings
[WARNING] Unrecognised tag: 'profile' (position: START_TAG seen ...    variables for plugins in the POM.\n   |\n   |-->\n   \t\t\t<profile>... @184:16)  @ D:\1win7\java\apache-maven-3.3.9-bin\apache-maven-3.3.9\conf\settings.xml, line 184, column 16
[WARNING] 
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building ADAM_2.10: Core 0.19.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-enforcer-plugin:1.0:enforce (enforce-versions) @ adam-core_2.10 ---
[INFO] 
[INFO] --- maven-enforcer-plugin:1.0:enforce (enforce-maven) @ adam-core_2.10 ---
[INFO] 
[INFO] --- build-helper-maven-plugin:1.9.1:add-source (add-source) @ adam-core_2.10 ---
[INFO] Source directory: D:\all\eclipse432\adam-2.10-0.19-git-bin\adam-2.10-0.19-git\adam-core\src\main\scala added.
[INFO] 
[INFO] --- scalariform-maven-plugin:0.1.4:format (default-cli) @ adam-core_2.10 ---
[INFO] Modified 0 of 159 .scala files
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ adam-core_2.10 ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] 
[INFO] --- scala-maven-plugin:3.2.2:compile (scala-compile-first) @ adam-core_2.10 ---
[WARNING]  Expected all dependencies to require Scala version: 2.10.4
[WARNING]  org.bdgenomics.utils:utils-misc_2.10:0.2.4 requires scala version: 2.10.4
[WARNING]  org.bdgenomics.utils:utils-misc_2.10:0.2.4 requires scala version: 2.10.4
[WARNING]  org.bdgenomics.utils:utils-metrics_2.10:0.2.4 requires scala version: 2.10.4
[WARNING]  org.bdgenomics.utils:utils-io_2.10:0.2.4 requires scala version: 2.10.4
[WARNING]  org.bdgenomics.utils:utils-cli_2.10:0.2.4 requires scala version: 2.10.4
[WARNING]  org.scoverage:scalac-scoverage-plugin_2.10:1.1.1 requires scala version: 2.10.4
[WARNING]  org.bdgenomics.adam:adam-core_2.10:0.19.0 requires scala version: 2.10.4
[WARNING]  com.twitter:chill_2.10:0.5.0 requires scala version: 2.10.4
[WARNING]  com.typesafe.akka:akka-remote_2.10:2.3.11 requires scala version: 2.10.4
[WARNING]  com.typesafe.akka:akka-actor_2.10:2.3.11 requires scala version: 2.10.4
[WARNING]  com.typesafe.akka:akka-slf4j_2.10:2.3.11 requires scala version: 2.10.4
[WARNING]  org.apache.spark:spark-core_2.10:1.5.2 requires scala version: 2.10.4
[WARNING]  org.json4s:json4s-jackson_2.10:3.2.10 requires scala version: 2.10.4
[WARNING]  org.json4s:json4s-core_2.10:3.2.10 requires scala version: 2.10.4
[WARNING]  org.json4s:json4s-ast_2.10:3.2.10 requires scala version: 2.10.4
[WARNING]  org.json4s:json4s-core_2.10:3.2.10 requires scala version: 2.10.0
[WARNING] Multiple versions of scala libraries detected!
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-compiler-plugin:3.3:compile (default-compile) @ adam-core_2.10 ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- build-helper-maven-plugin:1.9.1:add-test-source (add-test-source) @ adam-core_2.10 ---
[INFO] Test Source directory: D:\all\eclipse432\adam-2.10-0.19-git-bin\adam-2.10-0.19-git\adam-core\src\test\scala added.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ adam-core_2.10 ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 65 resources
[INFO] 
[INFO] --- scala-maven-plugin:3.2.2:testCompile (scala-test-compile-first) @ adam-core_2.10 ---
[WARNING]  Expected all dependencies to require Scala version: 2.10.4
[WARNING]  org.bdgenomics.utils:utils-misc_2.10:0.2.4 requires scala version: 2.10.4
[WARNING]  org.bdgenomics.utils:utils-misc_2.10:0.2.4 requires scala version: 2.10.4
[WARNING]  org.bdgenomics.utils:utils-metrics_2.10:0.2.4 requires scala version: 2.10.4
[WARNING]  org.bdgenomics.utils:utils-io_2.10:0.2.4 requires scala version: 2.10.4
[WARNING]  org.bdgenomics.utils:utils-cli_2.10:0.2.4 requires scala version: 2.10.4
[WARNING]  org.scoverage:scalac-scoverage-plugin_2.10:1.1.1 requires scala version: 2.10.4
[WARNING]  org.bdgenomics.adam:adam-core_2.10:0.19.0 requires scala version: 2.10.4
[WARNING]  com.twitter:chill_2.10:0.5.0 requires scala version: 2.10.4
[WARNING]  com.typesafe.akka:akka-remote_2.10:2.3.11 requires scala version: 2.10.4
[WARNING]  com.typesafe.akka:akka-actor_2.10:2.3.11 requires scala version: 2.10.4
[WARNING]  com.typesafe.akka:akka-slf4j_2.10:2.3.11 requires scala version: 2.10.4
[WARNING]  org.apache.spark:spark-core_2.10:1.5.2 requires scala version: 2.10.4
[WARNING]  org.json4s:json4s-jackson_2.10:3.2.10 requires scala version: 2.10.4
[WARNING]  org.json4s:json4s-core_2.10:3.2.10 requires scala version: 2.10.4
[WARNING]  org.json4s:json4s-ast_2.10:3.2.10 requires scala version: 2.10.4
[WARNING]  org.json4s:json4s-core_2.10:3.2.10 requires scala version: 2.10.0
[WARNING] Multiple versions of scala libraries detected!
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-compiler-plugin:3.3:testCompile (default-testCompile) @ adam-core_2.10 ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ adam-core_2.10 ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- scalatest-maven-plugin:1.0:test (test) @ adam-core_2.10 ---
[36mDiscovery starting.[0m
[36mDiscovery completed in 50 seconds, 526 milliseconds.[0m
[36mRun starting. Expected test count is: 7[0m
[32mAlignmentRecordConverterSuite:[0m
[32m- testing the fields in a converted ADAM Read[0m
[32m- converting a read with null quality is OK[0m
[32m- convert a read to fastq[0m
[32m- reverse complement reads when converting to fastq[0m
[32m- converting to fastq with unmapped reads[0m
[32m- converting a fragment with no alignments should yield unaligned reads[0m
[32m- converting a fragment with alignments should restore the alignments[0m
[36mRun completed in 54 seconds, 557 milliseconds.[0m
[36mTotal number of tests run: 7[0m
[36mSuites: completed 2, aborted 0[0m
[36mTests: succeeded 7, failed 0, canceled 0, ignored 0, pending 0[0m
[32mAll tests passed.[0m
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:44 min
[INFO] Finished at: 2016-05-13T19:31:57+08:00
[INFO] Final Memory: 24M/207M
[INFO] ------------------------------------------------------------------------

附录测试源文件:

/**
 * Licensed to Big Data Genomics (BDG) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership.  The BDG licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.bdgenomics.adam.converters

import htsjdk.samtools.{ SamReaderFactory, SAMRecord }
import java.io.File
import org.bdgenomics.adam.models.{
  RecordGroupDictionary,
  RecordGroup,
  SAMFileHeaderWritable,
  SequenceDictionary,
  SequenceRecord
}
import org.bdgenomics.formats.avro.{
  AlignmentRecord,
  Contig,
  Fragment
}
import org.scalatest.FunSuite
import scala.collection.JavaConversions._

class AlignmentRecordConverterSuite extends FunSuite {

  // allocate converters
  val adamRecordConverter = new AlignmentRecordConverter

  def makeRead(start: Long, cigar: String, mdtag: String, length: Int, id: Int = 0, nullQuality: Boolean = false): AlignmentRecord = {
    val sequence: String = "A" * length
    val builder = AlignmentRecord.newBuilder()
      .setReadName("read" + id.toString)
      .setStart(start)
      .setReadMapped(true)
      .setCigar(cigar)
      .setSequence(sequence)
      .setReadNegativeStrand(false)
      .setMapq(60)
      .setMismatchingPositions(mdtag)
      .setOldPosition(12)
      .setOldCigar("2^AAA3")

    if (!nullQuality) {
      builder.setQual(sequence) // no typo, we just don't care
    }

    builder.build()
  }

  test("testing the fields in a converted ADAM Read") {
    val adamRead = makeRead(3L, "2M3D2M", "2^AAA2", 4)

    // add reference details
    adamRead.setRecordGroupName("record_group")
    adamRead.setRecordGroupSample("sample")
    adamRead.setContig(Contig.newBuilder()
      .setContigName("referencetest")
      .build())
    adamRead.setMateContig(Contig.newBuilder()
      .setContigName("matereferencetest")
      .setContigLength(6L)
      .setReferenceURL("test://chrom1")
      .build())
    adamRead.setMateAlignmentStart(6L)

    // make sequence dictionary
    val seqRecForDict = SequenceRecord("referencetest", 5, "test://chrom1")
    val dict = SequenceDictionary(seqRecForDict)

    //make read group dictionary
    val readGroup = new RecordGroup(adamRead.getRecordGroupSample(), adamRead.getRecordGroupName())
    val readGroups = new RecordGroupDictionary(Seq(readGroup))

    // convert read
    val toSAM = adamRecordConverter.convert(adamRead,
      SAMFileHeaderWritable(adamRecordConverter.createSAMHeader(dict,
        readGroups)),
      readGroups)

    // validate conversion
    val sequence = "A" * 4
    assert(toSAM.getReadName === ("read" + 0.toString))
    assert(toSAM.getAlignmentStart === 4)
    assert(toSAM.getReadUnmappedFlag === false)
    assert(toSAM.getCigarString === "2M3D2M")
    assert(toSAM.getReadString === sequence)
    assert(toSAM.getReadNegativeStrandFlag === false)
    assert(toSAM.getMappingQuality === 60)
    assert(toSAM.getBaseQualityString === sequence)
    assert(toSAM.getAttribute("MD") === "2^AAA2")
    assert(toSAM.getIntegerAttribute("OP") === 13)
    assert(toSAM.getStringAttribute("OC") === "2^AAA3")
    //make sure that we didn't set the SM attribute.
    //issue #452 https://github.com/bigdatagenomics/adam/issues/452
    assert(toSAM.getAttribute("SM") === null)
    assert(toSAM.getHeader().getReadGroup("record_group").getSample() === "sample")
  }

  test("converting a read with null quality is OK") {
    val adamRead = makeRead(3L, "2M3D2M", "2^AAA2", 4, nullQuality = true)

    // add reference details
    adamRead.setRecordGroupName("record_group")
    adamRead.setRecordGroupSample("sample")
    adamRead.setContig(Contig.newBuilder()
      .setContigName("referencetest")
      .build())
    adamRead.setMateContig(Contig.newBuilder()
      .setContigName("matereferencetest")
      .setContigLength(6L)
      .setReferenceURL("test://chrom1")
      .build())
    adamRead.setMateAlignmentStart(6L)

    // make sequence dictionary
    val seqRecForDict = SequenceRecord("referencetest", 5, "test://chrom1")
    val dict = SequenceDictionary(seqRecForDict)

    //make read group dictionary
    val readGroup = new RecordGroup(adamRead.getRecordGroupSample(), adamRead.getRecordGroupName())
    val readGroups = new RecordGroupDictionary(Seq(readGroup))

    // convert read
    val toSAM = adamRecordConverter.convert(adamRead,
      SAMFileHeaderWritable(adamRecordConverter.createSAMHeader(dict,
        readGroups)),
      readGroups)

    // validate conversion
    val sequence = "A" * 4
    assert(toSAM.getReadName === ("read" + 0.toString))
    assert(toSAM.getAlignmentStart === 4)
    assert(toSAM.getReadUnmappedFlag === false)
    assert(toSAM.getCigarString === "2M3D2M")
    assert(toSAM.getReadString === sequence)
    assert(toSAM.getReadNegativeStrandFlag === false)
    assert(toSAM.getMappingQuality === 60)
    assert(toSAM.getBaseQualityString === "*")
    assert(toSAM.getAttribute("MD") === "2^AAA2")
    assert(toSAM.getIntegerAttribute("OP") === 13)
    assert(toSAM.getStringAttribute("OC") === "2^AAA3")
    //make sure that we didn't set the SM attribute.
    //issue #452 https://github.com/bigdatagenomics/adam/issues/452
    assert(toSAM.getAttribute("SM") === null)
    assert(toSAM.getHeader().getReadGroup("record_group").getSample() === "sample")
  }

  test("convert a read to fastq") {
    val adamRead = AlignmentRecord.newBuilder()
      .setSequence("ACACCAACATG")
      .setQual(".+**.+;:**.")
      .setReadName("thebestread")
      .build()

    val fastq = adamRecordConverter.convertToFastq(adamRead)
      .toString
      .split('\n')

    assert(fastq(0) === "@thebestread")
    assert(fastq(1) === "ACACCAACATG")
    assert(fastq(2) === "+")
    assert(fastq(3) === ".+**.+;:**.")
  }

  def getSAMRecordFromReadName(readName: String): (AlignmentRecord, AlignmentRecord) = {
    val samToADAMConverter = new SAMRecordConverter
    val SAMTestFile = new File(getClass.getClassLoader.getResource("bqsr1.sam").getFile)
    val newSAMReader = SamReaderFactory.makeDefault().open(SAMTestFile)

    // Obtain SAMRecord
    val newSAMRecord = newSAMReader.iterator().dropWhile(r => r.getReadName != readName)
    val newSequenceRecord = SequenceRecord("22", 51304566)
    val newSequenceDictionary = SequenceDictionary(newSequenceRecord)
    val firstRecord = samToADAMConverter.convert(newSAMRecord.next(), newSequenceDictionary, new RecordGroupDictionary(Seq()))
    val secondRecord = samToADAMConverter.convert(newSAMRecord.next(), newSequenceDictionary, new RecordGroupDictionary(Seq()))
    (firstRecord, secondRecord)
  }

  test("reverse complement reads when converting to fastq") {

    // SRR062634.10022079      83      22      16082719        0       5S95M   =       16082635        -179    
    // AAGTAGCTGGGACTACACGCACGCACCACCATGCCTGGCTAATTTTTGTATTTTTAGTAGAGATGAGGTTTCACCATATTGGCCAGGCTGGTTTTGAATT    
    // #####EB5BB<840&:2?>A?-AC8=,5@AABCB?CEDBDC@6BB,CA0CB,B-DEDEDEDEA:D?DE5EBEC?E?5?D:AEEEDEDDEEE=BEEBDD-?    
    // RG:Z:SRR062634  XC:i:95 XT:A:R  NM:i:2  SM:i:0  AM:i:0  X0:i:3  X1:i:0  XM:i:2  XO:i:0  XG:i:0  MD:Z:15G0T78    
    // XA:Z:GL000244.1,+31092,100M,2;14,+19760216,100M,2;

    val (firstRecord, secondRecord) = getSAMRecordFromReadName("SRR062634.10022079")

    assert(firstRecord.getReadInFragment === 1)
    assert(secondRecord.getReadInFragment === 0)

    val firstRecordFastq = adamRecordConverter.convertToFastq(firstRecord, maybeAddSuffix = true)
      .toString
      .split('\n')

    assert(firstRecordFastq(0) === "@SRR062634.10022079/2")
    assert(firstRecordFastq(1) === "CTGGAGTGCAGTGGCATGATTTCAGCTCACTGTCGTCTCTGCCTCCCTGACTCAAGTGATTCTCCTGCCTCAGCCTCCCACGTCGCTCGGACTCCACGCC")
    assert(firstRecordFastq(2) === "+")
    assert(firstRecordFastq(3) === "A:=D5D5E?D?DDD:.@@@@=?EE=DADDB@D=DD??ED=:CCCC?D:E=EEB=-C>C=@=EEEEB5EC-?A>=C-C?DC+34+4A>-?5:=/-A=@>>:")

    val secondRecordFastq = adamRecordConverter.convertToFastq(secondRecord, maybeAddSuffix = true)
      .toString
      .split('\n')

    assert(secondRecordFastq(0) === "@SRR062634.10022079/1")
    assert(secondRecordFastq(1) === "AATTCAAAACCAGCCTGGCCAATATGGTGAAACCTCATCTCTACTAAAAATACAAAAATTAGCCAGGCATGGTGGTGCGTGCGTGTAGTCCCAGCTACTT")
    assert(secondRecordFastq(2) === "+")
    assert(secondRecordFastq(3) === "?-DDBEEB=EEEDDEDEEEA:D?5?E?CEBE5ED?D:AEDEDEDED-B,BC0AC,BB6@CDBDEC?BCBAA@5,=8CA-?A>?2:&048<BB5BE#####")

  }

  test("converting to fastq with unmapped reads") {
    //SRR062634.10448889      117     22      16079761        0       *       =       16079761        0       
    // TTTCTTTCTTTTATATATATATACACACACACACACACACACACACATATATGTATATATACACGTATATGTATGTATATATGTATATATACACGTATAT    
    // @DF>C;FDC=EGEGGEFDGEFDD?DFDEEGFGFGGGDGGGGGGGEGGGGFGGGFGGGGGGFGGFGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG    
    // RG:Z:SRR062634

    val (secondRecord, firstRecord) = getSAMRecordFromReadName("SRR062634.10448889")

    assert(firstRecord.getReadInFragment === 1)
    assert(secondRecord.getReadInFragment === 0)

    val firstRecordFastq = adamRecordConverter.convertToFastq(firstRecord, maybeAddSuffix = true)
      .toString
      .split('\n')

    assert(firstRecord.getReadMapped)
    assert(firstRecord.getReadNegativeStrand)
    assert(firstRecordFastq(0) === "@SRR062634.10448889/2")
    assert(firstRecordFastq(1) === "ACCTGTCTCAGCCTCCCAAAGTGCTGCGATTACAGTCATGAGCCACCGCACTTGGCTGGGTTTTCGTTTTCTTTCTTTTATATATATATACACACACACA")
    assert(firstRecordFastq(2) === "+")
    assert(firstRecordFastq(3) === "GGGGGGGGGGGGGGGGGGGGGEGGGGGGGGGGGGGGGGGGGGGGGGGFGEGEEDGGFDF?AEEEBDADEEDEEE;DFC@'B:B=B=B=BADCBCBCA=DA")

    val secondRecordFastq = adamRecordConverter.convertToFastq(secondRecord, maybeAddSuffix = true)
      .toString
      .split('\n')

    assert(!secondRecord.getReadMapped)
    assert(secondRecord.getReadNegativeStrand)
    assert(secondRecordFastq(0) === "@SRR062634.10448889/1")
    assert(secondRecordFastq(1) === secondRecord.getSequence)
    assert(secondRecordFastq(2) === "+")
    assert(secondRecordFastq(3) === secondRecord.getQual)
  }

  test("converting a fragment with no alignments should yield unaligned reads") {
    val alignments = List(
      AlignmentRecord.newBuilder()
        .setSequence("ACCCACAGTA")
        .setQual("**********")
        .setReadInFragment(0)
        .setReadName("testRead")
        .setReadPaired(true)
        .build(),
      AlignmentRecord.newBuilder()
        .setSequence("GGGAAACCCTTT")
        .setQual(";;;;;;......")
        .setReadName("testRead")
        .setReadInFragment(1)
        .setReadPaired(true)
        .build())

    val fragment = Fragment.newBuilder()
      .setReadName("testRead")
      .setAlignments(seqAsJavaList(alignments))
      .build()

    val reads = adamRecordConverter.convertFragment(fragment)
    assert(reads.size === 2)

    val read1 = reads.find(_.getReadInFragment == 0)
    assert(read1.isDefined)
    assert(read1.get.getSequence === "ACCCACAGTA")
    assert(read1.get.getQual() === "**********")
    assert(read1.get.getReadName === "testRead")
    assert(!read1.get.getReadMapped)
    assert(read1.get.getReadPaired)

    val read2 = reads.find(_.getReadInFragment == 1)
    assert(read2.isDefined)
    assert(read2.get.getSequence === "GGGAAACCCTTT")
    assert(read2.get.getQual() === ";;;;;;......")
    assert(read2.get.getReadName === "testRead")
    assert(!read2.get.getReadMapped)
    assert(read2.get.getReadPaired)
  }

  test("converting a fragment with alignments should restore the alignments") {
    val alignments = List(AlignmentRecord.newBuilder()
      .setReadMapped(true)
      .setContig(Contig.newBuilder()
        .setContigName("1")
        .build())
      .setStart(10L)
      .setEnd(20L)
      .setReadName("testRead")
      .setCigar("10M")
      .setReadNegativeStrand(true)
      .setSequence("TACTGTGGGT")
      .setQual("?????*****")
      .build())
    val fragment = Fragment.newBuilder()
      .setReadName("testRead")
      .setAlignments(seqAsJavaList(alignments))
      .build()

    val reads = adamRecordConverter.convertFragment(fragment)
    assert(reads.size === 1)
    val read = reads.head

    assert(read.getReadName === "testRead")
    assert(read.getReadInFragment === 0)
    assert(read.getReadMapped)
    assert(read.getReadNegativeStrand)
    assert(read.getStart === 10L)
    assert(read.getEnd === 20L)
    assert(read.getCigar === "10M")
    assert(read.getSequence === "TACTGTGGGT")
    assert(read.getQual === "?????*****")
    assert(read.getContig.getContigName === "1")
  }
}

参考:
【1】 http://www.scalatest.org/user_guide/using_the_scalatest_maven_plugin
【2】http://www.scalatest.org/user_guide/using_scalatest_with_eclipse
【3】https://github.com/scalatest/scalatest-eclipse-plugin
【4】http://blog.csdn.net/u011414200/article/details/49333757

 类似资料: