typo3-solr-solrfal

name	typo3-solr/SOLRFAL
description	File indexing with EXT:solrfal and Apache Tika for TYPO3. Index PDFs, DOCX, XLSX and other file formats into Solr for full-text file search. Setup, configuration, Tika Server, troubleshooting.

EXT:solrfal -- File Indexing for Solr

Funding Extension: EXT:solrfal is a funding contribution extension, not open source. It is available to EB partners and via the dkd shop.

1. What is solrfal?

EXT:solrfal enables indexing of TYPO3 FAL (File Abstraction Layer) files into Apache Solr. It extracts text content from binary files (PDFs, Word documents, spreadsheets, etc.) using Apache Tika, and adds them as searchable documents in Solr.

graph LR
    subgraph typo3 [TYPO3 CMS]
        FAL["FAL File<br/>(PDF, DOCX, ...)"]
        SolrFal["EXT:solrfal<br/>(Queue + Config)"]
    end
    subgraph tika [Apache Tika]
        Extract["Text Extraction<br/>+ Metadata"]
    end
    subgraph solr [Apache Solr]
        Doc["Solr Document<br/>(title, content, url, ...)"]
    end
    FAL --> SolrFal
    SolrFal --> Extract
    Extract --> SolrFal
    SolrFal -->|POST document| Doc

When files are indexed, search results display differently from page results:

• File title and metadata (MIME type, size, author)
• Link to download the file
• References to pages where the file is used

2. Prerequisites

Component	Version	Required?
EXT:solr	13.1	Yes
EXT:solrfal	13.0	Yes
EXT:tika	13.1	Yes
Apache Tika Server	3.2.2+	Yes (CVE-2025-54988 fix)
EB Partner access	--	Yes (funding extension)

3. Installation

composer require apache-solr-for-typo3/solrfal
composer require apache-solr-for-typo3/tika

EXT:solrfal is not on Packagist. It is distributed via the dkd Composer repository. Add the repository URL provided with your EB partner credentials to composer.json.

4. Tika Server Setup

The recommended approach is running Tika Server as a Docker container:

services:
  tika:
    image: apache/tika:3.2.2
    ports:
      - "9998:9998"
    restart: unless-stopped

For DDEV, create .ddev/docker-compose.tika.yaml:

services:
  tika:
    container_name: ddev-${DDEV_SITENAME}-tika
    image: apache/tika:3.2.2
    restart: "no"
    labels:
      com.ddev.site-name: ${DDEV_SITENAME}
      com.ddev.approot: $DDEV_APPROOT
    expose:
      - "9998"
    environment:
      - VIRTUAL_HOST=$DDEV_HOSTNAME

Configure EXT:tika

1. Open TYPO3 Backend -> Admin Tools -> Settings -> Extension Configuration -> tika
2. Select "Tika Server" as the extractor in the General tab
3. Enter connection info in the Server tab:
   • Host: tika (Docker service name) or localhost
   • Port: 9998
4. Click "Check if it works" to verify the connection

Alternative: Solr Cell

If you don't want a separate Tika container, use Solr Cell (Tika built into Solr):

1. In EXT:tika settings, select "Solr Server" as the extractor
2. Configure the Solr connection (same as EXT:solr)

Limitation: Solr Cell uses the Tika version bundled with Solr, which may be older than the standalone Tika Server.

5. Configuration

Enable File Indexing

File indexing is configured via the EXT:solr Index Queue, attached to content elements or records that reference files:

plugin.tx_solr.index.queue {
    pages {
        attachments {
            fields = media
        }
    }

    news {
        attachments {
            fields = related_files
        }
    }
}

The attachments.fields setting specifies which FAL relation fields contain files to index.

Supported File Types

Tika can extract text from ~600 file formats. The most common:

Format	MIME Type	Extracted Content
PDF	application/pdf	Text, metadata, embedded images (text only)
DOCX	application/vnd.openxmlformats-officedocument.wordprocessingml.document	Text, metadata
XLSX	application/vnd.openxmlformats-officedocument.spreadsheetml.sheet	Cell content, metadata
PPTX	application/vnd.openxmlformats-officedocument.presentationml.presentation	Slide text, notes
ODT	application/vnd.oasis.opendocument.text	Text, metadata
RTF	application/rtf	Text
TXT	text/plain	Raw text
HTML	text/html	Stripped text
MSG	application/vnd.ms-outlook	Email body, metadata

File Size Limits

Large files can cause timeouts or memory issues. Configure limits in EXT:tika extension settings:

• Maximum file size: Set a reasonable limit (e.g., 20 MB) to prevent Tika timeouts
• Tika timeout: Increase the server timeout for large documents

6. Index Queue for Files

Files enter the Index Queue when:

• A content element or record with file references is created/edited
• The file reference field is in the attachments.fields configuration
• The file type is extractable by Tika

Files are processed by the same "Index Queue Worker" scheduler task as regular records.

How File Documents Look in Solr

File documents in Solr contain:

• type: tx_solrfal_file (or similar, depending on configuration)
• title: File name or extracted title metadata
• content: Extracted text content from Tika
• url: Download URL for the file
• fileReferenceUid_intS: UID of the FAL file reference
• mimeType_stringS: MIME type of the file
• Standard fields: siteHash, access, pid, etc.

7. Troubleshooting

Tika Connection Issues

Symptom	Fix
"Connection refused" to Tika	Check Docker container is running: docker ps | grep tika
Timeout on large files	Increase Tika Server timeout, reduce max file size
"Unsupported Media Type"	File format not supported by Tika, check MIME type
Empty content extracted	File may be scanned image (needs OCR), or password-protected

DDEV-Specific

In DDEV, the Tika service name is tika (use this as hostname). Test connectivity:

ddev exec curl -s http://tika:9998/tika

Files Not Appearing in Search Results

1. Check attachments.fields in TypoScript includes the correct FAL field name
2. Verify the file type is supported by Tika
3. Check the Index Queue for file items and error messages
4. Query Solr Admin: q=*:*&fq=type:tx_solrfal_file&fl=uid,title,content,mimeType_stringS
5. Ensure the Scheduler worker has processed file items

Performance

• Index files during off-peak hours (Tika extraction is CPU-intensive)
• Use the two-pass indexing approach: index records first, then process files
• Monitor Tika Server memory usage for large document collections

8. Testing

Verify Tika Extraction

Test Tika directly:

# Send a PDF to Tika and get extracted text
curl -T /path/to/test.pdf http://tika:9998/tika --header "Accept: text/plain"

# Get metadata
curl -T /path/to/test.pdf http://tika:9998/meta --header "Accept: application/json"

Verify Solr Documents

After indexing, check file documents in Solr Admin:

q=*:*&fq=type:tx_solrfal_file&rows=5&fl=uid,title,content,mimeType_stringS,url

Functional Test

<?php

declare(strict_types=1);

namespace MyVendor\MyExt\Tests\Functional;

use TYPO3\TestingFramework\Core\Functional\FunctionalTestCase;

final class FileIndexingTest extends FunctionalTestCase
{
    protected array $testExtensionsToLoad = [
        'apache-solr-for-typo3/solr',
        'apache-solr-for-typo3/tika',
        'apache-solr-for-typo3/solrfal',
    ];

    public function testPdfFileIsQueued(): void
    {
        $this->importCSVDataSet(__DIR__ . '/Fixtures/page_with_pdf.csv');

        // Verify the file appears in the index queue
        $queryBuilder = $this->getConnectionPool()
            ->getQueryBuilderForTable('tx_solr_indexqueue_item');
        $count = $queryBuilder
            ->count('uid')
            ->from('tx_solr_indexqueue_item')
            ->where(->()->(, ->()))
            ->()
            ->();

        ::(, , );
    }
}