Convert. html log with nested tables to. csv file

Question

I'm trying to convert one. html file containing logs in tabular form, which has nested tables. I'm converting it to . csv file. There is an error report in one of the columns and as a new table in that column. I want to convert the entire table to plain text. Tried using beautifulsoup in python to achieve this but no luck yet. Data in a nested table is spread across all columns of the parent table, rather than being fixed in the original columns. Is there anything I can do?

Using python with the beautifulsoup library is not giving the desired output

Answer 1

Converting HTML files with nested tables to CSV while preserving the structure can be a bit difficult. BeautifulSoup is a great library for parsing HTML, but it may require additional operations to properly handle nested tables.

To get the desired output, BeautifulSoup can be used with some custom Python code to parse the HTML, extract the data, and organize it correctly into CSV format. Here's a step-by-step method to help you achieve this goal:

Use BeautifulSoup to parse HTML files.

1. Find the parent table and extract its header.
2. Find all rows in the parent table.
3. For each row, find the nested table in the relevant column (if it exists).
4. Extract data from a nested table and append it to the corresponding cells in the parent table.

Here is a Python code snippet to help you get started:

from bs4 import BeautifulSoup
import csv

def extract_nested_table_data(table_cell):
    # Helper function to extract the data from a nested table cell
    nested_table = table_cell.find('table')
    if not nested_table:
        return ''

    # Process the nested table and extract its data as plain text
    nested_rows = nested_table.find_all('tr')
    nested_data = []
    for row in nested_rows:
        nested_cells = row.find_all(['td', 'th'])
        nested_data.append([cell.get_text(strip=True) for cell in nested_cells])
    
    # Convert nested_data to a formatted plain text representation
    nested_text = '\n'.join(','.join(row) for row in nested_data)
    return nested_text

def convert_html_to_csv(html_filename, csv_filename):
    with open(html_filename, 'r', encoding='utf-8') as html_file:
        soup = BeautifulSoup(html_file, 'html.parser')

        parent_table = soup.find('table')
        headers = [header.get_text(strip=True) for header in parent_table.find_all('th')]

        with open(csv_filename, 'w', newline='', encoding='utf-8') as csv_file:
            csv_writer = csv.writer(csv_file)
            csv_writer.writerow(headers)

            rows = parent_table.find_all('tr')
            for row in rows[1:]:  # Skipping the header row
                cells = row.find_all(['td', 'th'])
                row_data = [cell.get_text(strip=True) for cell in cells]

                # Extract data from nested table (if it exists) and append to the row
                for idx, cell in enumerate(cells):
                    nested_data = extract_nested_table_data(cell)
                    row_data[idx] += nested_data

                csv_writer.writerow(row_data)

if __name__ == '__main__':
    html_filename = 'input.html'
    csv_filename = 'output.csv'
    convert_html_to_csv(html_filename, csv_filename)

This code assumes that your nested table data is comma-separated. If it's not, you may need to adjust the separator accordingly. Additionally, consider other delimiters if your nested table contains commas.

Remember that handling complex HTML structures may require further adjustments to this code, depending on the specifics of your data. Nonetheless, this should serve as a good starting point to tackle the task.