将SQL ResultSet视为Scala流

当我查询数据库并收到(仅转发,只读)ResultSet时,ResultSet的作用类似于数据库行的列表。

我正在尝试找到一种将此ResultSet像Scala Stream一样对待的方法。这将允许诸如filtermap之类的操作,而不会消耗大量的RAM。

我实现了尾递归方法来提取单个项目,但这要求所有项目都同时在内存中,如果ResultSet非常大,则会出现问题:

// Iterate through the result set and gather all of the String values into a list
// then return that list
@tailrec
def loop(resultSet: ResultSet,
         accumulator: List[String] = List()): List[String] = {
  if (!resultSet.next) accumulator.reverse
  else {
    val value = resultSet.getString(1)
    loop(resultSet, value +: accumulator)
  }
}
Ralph asked 2020-08-10T09:42:54Z
8个解决方案
73 votes

我没有测试,但是为什么不起作用?

new Iterator[String] {
  def hasNext = resultSet.next()
  def next() = resultSet.getString(1)
}.toStream
elbowich answered 2020-08-10T09:42:59Z
10 votes

实用功能@elbowich的答案:

def results[T](resultSet: ResultSet)(f: ResultSet => T) = {
  new Iterator[T] {
    def hasNext = resultSet.next()
    def next() = f(resultSet)
  }
}

允许您使用类型推断。 例如。:

stmt.execute("SELECT mystr, myint FROM mytable")

// Example 1:
val it = results(stmt.resultSet) {
  case rs => rs.getString(1) -> 100 * rs.getInt(2)
}
val m = it.toMap // Map[String, Int]

// Example 2:
val it = results(stmt.resultSet)(_.getString(1))
hraban answered 2020-08-10T09:43:24Z
8 votes

对于隐式类来说,这听起来像是一个绝佳的机会。 首先在某处定义隐式类:

import java.sql.ResultSet

object Implicits {

    implicit class ResultSetStream(resultSet: ResultSet) {

        def toStream: Stream[ResultSet] = {
            new Iterator[ResultSet] {
                def hasNext = resultSet.next()

                def next() = resultSet
            }.toStream
        }
    }
}

接下来,只需在执行查询并定义ResultSet对象的任何位置导入此隐式类即可:

import com.company.Implicits._

最后,使用toStream方法获取数据。 例如,获取所有ID,如下所示:

val allIds = resultSet.toStream.map(result => result.getInt("id"))
Jeroen Minnaert answered 2020-08-10T09:43:53Z
3 votes

我需要类似的东西。 基于elbowich的非常酷的答案,我将其包装了一下,然后返回结果而不是字符串(因此您可以获取任何列)

def resultSetItr(resultSet: ResultSet): Stream[ResultSet] = {
    new Iterator[ResultSet] {
      def hasNext = resultSet.next()
      def next() = resultSet
    }.toStream
  }

我需要访问表元数据,但这将适用于表行(可以执行stmt.executeQuery(sql)而不是md.getColumns):

 val md = connection.getMetaData()
 val columnItr = resultSetItr( md.getColumns(null, null, "MyTable", null))
      val columns = columnItr.map(col => {
        val columnType = col.getString("TYPE_NAME")
        val columnName = col.getString("COLUMN_NAME")
        val columnSize = col.getString("COLUMN_SIZE")
        new Column(columnName, columnType, columnSize.toInt, false)
      })
Greg answered 2020-08-10T09:44:18Z
2 votes

因为ResultSet只是一个下一步导航的可变对象,所以我们需要定义下一行的概念。 我们可以使用如下输入函数来做到这一点:

class ResultSetIterator[T](rs: ResultSet, nextRowFunc: ResultSet => T) 
extends Iterator[T] {

  private var nextVal: Option[T] = None

  override def hasNext: Boolean = {
    val ret = rs.next()
    if(ret) {
      nextVal = Some(nextRowFunc(rs))
    } else {
      nextVal = None
    }
    ret
  }

  override def next(): T = nextVal.getOrElse { 
    hasNext 
    nextVal.getOrElse( throw new ResultSetIteratorOutOfBoundsException 
  )}

  class ResultSetIteratorOutOfBoundsException extends Exception("ResultSetIterator reached end of list and next can no longer be called. hasNext should return false.")
}

编辑:转换为流或按上述方式进行其他操作。

Brendan answered 2020-08-10T09:44:42Z
0 votes

此实现虽然更长且更麻烦,但它与ResultSet契约更好地对应。 副作用已从hasNext(...)中移除,并移至next()中。

new Iterator[String] {
  private var available = resultSet.next()
  override def hasNext: Boolean = available
  override def next(): String = {
    val string = resultSet.getString(1)
    available = resultSet.next()
    string
  }
}
thoredge answered 2020-08-10T09:45:03Z
0 votes

我认为上述大多数实现都有不确定的30050513491523023020方法。 调用两次将光标移至第二行。 我建议使用类似的东西:

  new Iterator[ResultSet] {
    def hasNext = {
      !resultSet.isLast
    }
    def next() = {
      resultSet.next()
      resultSet
    }
  }
Matzz answered 2020-08-10T09:45:23Z
0 votes
Iterator.continually(rs.next())
  .takeWhile(identity)
  .map(_ => Model(
      id = rs.getInt("id"),
      text = rs.getString("text")
   ))
Sergey Alaev answered 2020-08-10T09:45:38Z
translate from https://stackoverflow.com:/questions/9636545/treating-an-sql-resultset-like-a-scala-stream