Skip to content

bug fix: truncate wide columns in print.data.table #7718 #7758

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
venom1204 wants to merge 2 commits into
base: master
Choose a base branch
from
+26 −9
Open

bug fix: truncate wide columns in print.data.table #7718 #7758

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
112f675
proper checks
venom1204 May 25, 2026
1af6897
news entry
venom1204 May 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions NEWS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@

9. Grouping operations on empty (0-row, 0-column) data.tables work as intended, [#7749](https://github.com/Rdatatable/data.table/issues/7749). Thanks @rickhelmus for the report and @MichaelChirico for the fix.

10. `print.data.table()` now correctly truncates long character columns and list-column summaries to avoid horizontal console overflow, [#7718](https://github.com/Rdatatable/data.table/issues/7718). When `datatable.prettyprint.char` is `NULL` (the default), the truncation limit is now dynamically calculated as `getOption("width") - 5L`. Thanks @tdhock for the report and @venom1204 for the fix.

### Notes

1. {data.table} now depends on R 3.5.0 (2018).
Expand Down
16 changes: 7 additions & 9 deletions R/print.data.table.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,12 +199,9 @@ has_format_method = function(x) {
}

format_col.default = function(x, ...) {
if (!is.null(dim(x)))
"<multi-column>"
else if (is.list(x))
vapply_1c(x, format_list_item, ...)
else
format(char.trunc(x), ...) # relevant to #37
Comment thread
tdhock marked this conversation as resolved.
if (!is.null(dim(x))) return("<multi-column>")
if (is.list(x)) x = vapply_1c(x, format_list_item, ...)
format(char.trunc(x), ...)
}

# #2842 -- different columns can have different tzone, so force usage in output
Expand Down Expand Up @@ -247,12 +244,13 @@ format_list_item.data.frame = function(x, ...) {
# Current implementation may have issues when dealing with strings that have combinations of full-width and half-width characters,
# if this becomes a problem in the future, we could consider string traversal instead.
char.trunc = function(x, trunc.char = getOption("datatable.prettyprint.char")) {
if (is.null(trunc.char)) trunc.char = getOption("width") - 5L
Copy link
Copy Markdown
Member

@joshhwuu joshhwuu May 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just curious, why 5L? and this looks like a magic number, could we use a constant instead?

Copy link
Copy Markdown
Contributor Author

@venom1204 venom1204 May 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my major reason of using 5L was because toby sir mentioned this would options(datatable.prettyprint.char=getOption("width")-5) be possible? in the issue

Copy link
Copy Markdown
Member

@joshhwuu joshhwuu May 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would 5L still make sense if we had a large table and the row label prefix consists of many more characters? I would argue against a magic number or even reading the row label prefix if we can here

Copy link
Copy Markdown
Contributor Author

@venom1204 venom1204 May 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you are right the 5L buffer fails once row labels grow bigger on a narrow consoles.
to fix this we can either

  • increase the constant biffer instead of using 5L
  • or we can pass the width dyanamically for precise fit.

Copy link
Copy Markdown
Member

@tdhock tdhock May 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you please provide a couple examples of when 5L works and does not work?
does is work for the examples I put in the original issue?

Copy link
Copy Markdown
Contributor Author

@venom1204 venom1204 May 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its working fine for your example ,i was accounting about the final width because the 5L bugffer doent not account all layout components (row labels,spacing,'...')
for ex

options(width=25, datatable.prettyprint.char=NULL) dt = data.table( x = rep("ABCDEFGHIJKLMNOPQRSTUVWXYZ", 1000000) )
print(dt, topn=1)
>1000000: ABCDEFGHIJKLMNOPQRST...

if you count the truncated data only then its behaving correctly , however if teh fully rendered line is considered then its excedding the 25 limit.

Copy link
Copy Markdown
Member

@tdhock tdhock May 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks. options(width) is documented on ?options as a limit on the whole line (the maximum number of columns on a line used in printing vectors, matrices and arrays, and when filling by cat), so in this example the output should be 25 characters,

1000000: ABCDEFGHIJKLM...

Copy link
Copy Markdown
Contributor Author

@venom1204 venom1204 May 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you guide me towards what could be the next step.

Copy link
Copy Markdown
Member

@tdhock tdhock May 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m not sure, I have not thought it through.

trunc.char = max(0L, suppressWarnings(as.integer(trunc.char[1L])), na.rm=TRUE)
if (!is.character(x) || trunc.char <= 0L) return(x)
nchar_width = nchar(x, 'width') # Check whether string is full-width or half-width, #5096
nchar_chars = nchar(x, 'char')
nchar_width = nchar(x, 'width', allowNA=TRUE) # Check whether string is full-width or half-width, #5096
nchar_chars = nchar(x, 'char', allowNA=TRUE)
is_full_width = nchar_width > nchar_chars
idx = !is.na(x) & pmin(nchar_width, nchar_chars) > trunc.char
idx = !is.na(x) & !is.na(nchar_width) & pmin(nchar_width, nchar_chars) > trunc.char
Copy link
Copy Markdown
Member

@joshhwuu joshhwuu May 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we use na.rm in pmin instead of a guard?

Copy link
Copy Markdown
Contributor Author

@venom1204 venom1204 May 26, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i tried using na.rm = TRUE in pmin but found it causes NA values to be incorrectly flagged for truncation which leads to NA becoming "NA..." (breaking test 2253.2) and invalid 'width' argument crashes in strtrim.

x[idx] = paste0(strtrim(x[idx], trunc.char * fifelse(is_full_width[idx], 2L, 1L)), "...")
x
}
Expand Down
17 changes: 17 additions & 0 deletions inst/tests/tests.Rraw
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21620,3 +21620,20 @@ test(2373.1, empty_dt[, 1, by = numeric()], data.table(numeric=numeric(), V1=num
test(2373.2, empty_dt[, 1L, by = numeric()], data.table(numeric=numeric(), V1=integer()))
test(2373.3, empty_dt[, .(x=1), by = .(b=numeric())], data.table(b=numeric(), x=numeric()))
test(2373.4, data.table()[, x := 1, by=numeric()], data.table(x=numeric()))

# issue 7718
# print.data.table truncates long character columns based on width
options(width=10, datatable.prettyprint.char=NULL)
test(2374.1, capture.output(print(data.table(x="12345678901234567890"))), output="12345...")
Comment on lines +21626 to +21627
Copy link
Copy Markdown
Member

@tdhock tdhock May 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please use test(options=) instead of options(…), here and below

Suggested change
options(width=10, datatable.prettyprint.char=NULL)
test(2374.1, capture.output(print(data.table(x="12345678901234567890"))), output="12345...")
test(2374.1, capture.output(print(data.table(x="12345678901234567890"))), output="12345...", options=list(width=10, datatable.prettyprint.char=NULL))

# list columns are truncated to fit console width
options(width=15, datatable.prettyprint.char=NULL)
test(2374.2, capture.output(print(data.table(L=list(1:20)))) , output="1,2,3,4,5,...")
# long strings are truncated while short strings are preserved
options(width=20, datatable.prettyprint.char=NULL)
test(2374.3, capture.output(print(data.table(x=c("short", "abcdefghijklmnopqrstuvwxyz")))), output="abcdefghijklmno...")
# large widths print full character values without truncation
options(width=200, datatable.prettyprint.char=NULL)
test(2374.4, capture.output(print(data.table(x="abcdefghijklmnopqrstuvwxyz"))), output="abcdefghijklmnopqrstuvwxyz")
# mixed numeric and character columns preserve formatting while truncating long text
options(width=20, datatable.prettyprint.char=NULL)
test(2374.5, capture.output(print(data.table(id=1L, score=99.12345, txt="abcdefghijklmnopqrstuvwxyz"))), output="abcdefghijklmno...")
Loading