Bug 565081 - .gitignore has no effect on commited file in subfolder - changed from 4.11.9.. to 5.x
Summary: .gitignore has no effect on commited file in subfolder - changed from 4.11.9....
Status: RESOLVED FIXED
Alias: None
Product: JGit
Classification: Technology
Component: JGit (show other bugs)
Version: 5.8   Edit
Hardware: All All
: P3 normal (vote)
Target Milestone: 5.9   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-07-09 05:45 EDT by Thomas Stock CLA
Modified: 2020-07-16 19:41 EDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Stock CLA 2020-07-09 05:45:11 EDT
jgit marks a commited file that was later added to gitignore as deleted.

here is a unit test that shows the problem

My local git version is 2.27.0

-------------------------------
package any;

import static org.junit.Assert.assertEquals;

import java.io.BufferedOutputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.io.OutputStream;
import java.util.List;

import org.eclipse.jgit.api.Git;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.api.errors.JGitInternalException;
import org.eclipse.jgit.diff.DiffEntry;
import org.eclipse.jgit.diff.DiffFormatter;
import org.eclipse.jgit.diff.RawTextComparator;
import org.eclipse.jgit.dircache.DirCacheIterator;
import org.eclipse.jgit.junit.RepositoryTestCase;
import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.treewalk.AbstractTreeIterator;
import org.eclipse.jgit.treewalk.FileTreeIterator;
import org.junit.Test;

public class TestDiffFor extends RepositoryTestCase {

  @Test
  public void testEmptyLevel() throws IOException, GitAPIException {
    File emptyFolder = new File(db.getWorkTree(), "empty");
    emptyFolder.mkdir();
    File emptyChildFolder = new File(emptyFolder, "empty");
    emptyChildFolder.mkdir();
    // mkdir -p empty/empty
    File keep = new File(emptyChildFolder, ".gitkeep");
    write(keep, "");
    // touch empty/empty/.gitkeep

    try (Git git = new Git(db)) {
      git.add().addFilepattern(".").call();
      // git add -A
      git.commit().setMessage("Initial commit").call();
      // git commit -am "Initial commit"
      // https://git-scm.com/docs/gitignore
      write(new File(db.getWorkTree(), ".gitignore"), "empty/*"); // maybe in jgit ".*" is valid in git "*"?
      // echo "empty/*" > .gitignore
      git.add().addFilepattern(".").call();
      // git add -A
      git.commit().setMessage("Added ignore").call();
      // git commit -am "Added ignore"
      assertEquals("", formatDiff(git));
      // works with 4.11.9.201909030838-r
      // fails with 5.0.0.201806131550-r, 5.1.13.202002110435-r, 5.8.0.202006091008-r
      // git diff # is empty
    }
  }

  private static String formatDiff(Git git) {
    OutputStream out = new ByteArrayOutputStream();
    Repository repo = git.getRepository();
    DiffFormatter diffFmt = new DiffFormatter(new BufferedOutputStream(out));
    diffFmt.setRepository(repo);
    diffFmt.setDiffComparator(RawTextComparator.WS_IGNORE_TRAILING);
    try {
      AbstractTreeIterator oldTree = new DirCacheIterator(repo.readDirCache());
      AbstractTreeIterator newTree = new FileTreeIterator(repo);

      List<DiffEntry> result = diffFmt.scan(oldTree, newTree);
      diffFmt.format(result);

      diffFmt.flush();
    } catch (IOException e) {
      throw new JGitInternalException(e.getMessage(), e);
    } finally {
      diffFmt.close();
    }

    return out.toString();
  }
}
Comment 1 Thomas Wolf CLA 2020-07-10 12:37:04 EDT
Correct. Manifests itself only when you compute a diff; git.getStatus().call().isClean() == true.

This is an API and behavior change in 5.0. To speed up traversing the file system, the FileTreeIterator by default skips ignored directories. There are
two ways to deal with this in cases like your example, where you have a tracked file inside an ignored folder.

You can use

  FileTreeIterator newTree = new FileTreeIterator(repo);
  newTree.setWalkIgnoredDirectories(true);

This will walk all directories.

Or (in general, not in the diff case) you should be able to do

  TreeWalk walk = new TreeWalk(repo);
  AbstractTreeIterator oldTree = new DirCacheIterator(repo.readDirCache());
  FileTreeIterator newTree = new FileTreeIterator(repo);
  int dirCacheIndex = walk.addTree(oldTree);
  walk.addTree(newTree);
  newTree.setDirCacheIterator(walk, dirCacheIndex);
  
and then walk the tree. This will skip ignored directories unless they contain tracked resources.

Unfortunately this second way is not available for DiffFormat.scan(), which takes the two trees and then constructs its own walk. DiffFormat.scan() must account for the case where one of the two iterators is a WorkingTreeIterator and set up the iterator and the walk correctly in this case.
Comment 2 Eclipse Genie CLA 2020-07-11 09:33:06 EDT
New Gerrit change created: https://git.eclipse.org/r/c/jgit/jgit/+/166183